{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyO8x5f3Rk8pmK/cjrsBFKcD",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DanielWarfield1/MLWritingAndResearch/blob/main/IntroAIMNIST.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Dense NN on MNIST\n",
        "The following code is a simplified example of a model training and testing on MNIST. The objective is to improve this model, somehow, to be better."
      ],
      "metadata": {
        "id": "ZGPUtxKZ6vC9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Setup"
      ],
      "metadata": {
        "id": "XSrqv8GB7pL-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Downloading the MNIST training and testing data. There's a lot of ways to\n",
        "#get this data, this is just one that I found on Google.\n",
        "from torchvision import datasets\n",
        "from torchvision.transforms import ToTensor\n",
        "\n",
        "train_data = datasets.MNIST(\n",
        "    root = 'data',\n",
        "    train = True,\n",
        "    transform = ToTensor(),\n",
        "    download = True,\n",
        ")\n",
        "test_data = datasets.MNIST(\n",
        "    root = 'data',\n",
        "    train = False,\n",
        "    transform = ToTensor()\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ux5z-3Nc7LP8",
        "outputId": "6e11c415-2214-4eb2-f172-5c8533062f20"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 9912422/9912422 [00:00<00:00, 29771556.88it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 28881/28881 [00:00<00:00, 1285668.58it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 1648877/1648877 [00:00<00:00, 8261726.67it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Failed to download (trying next):\n",
            "HTTP Error 403: Forbidden\n",
            "\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 4542/4542 [00:00<00:00, 6700854.30it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Exploring the Data"
      ],
      "metadata": {
        "id": "Y0KvsTZK7qud"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "type(train_data)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "OZ8k_0dAEdkd",
        "outputId": "2fabeffd-b287-4da7-d128-a4d1d590bfcc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torchvision.datasets.mnist.MNIST"
            ],
            "text/html": [
              "<div style=\"max-width:800px; border: 1px solid var(--colab-border-color);\"><style>\n",
              "      pre.function-repr-contents {\n",
              "        overflow-x: auto;\n",
              "        padding: 8px 12px;\n",
              "        max-height: 500px;\n",
              "      }\n",
              "\n",
              "      pre.function-repr-contents.function-repr-contents-collapsed {\n",
              "        cursor: pointer;\n",
              "        max-height: 100px;\n",
              "      }\n",
              "    </style>\n",
              "    <pre style=\"white-space: initial; background:\n",
              "         var(--colab-secondary-surface-color); padding: 8px 12px;\n",
              "         border-bottom: 1px solid var(--colab-border-color);\"><b>torchvision.datasets.mnist.MNIST</b><br/>def __init__(root: Union[str, Path], train: bool=True, transform: Optional[Callable]=None, target_transform: Optional[Callable]=None, download: bool=False) -&gt; None</pre><pre class=\"function-repr-contents function-repr-contents-collapsed\" style=\"\"><a class=\"filepath\" style=\"display:none\" href=\"#\">/usr/local/lib/python3.10/dist-packages/torchvision/datasets/mnist.py</a>`MNIST &lt;http://yann.lecun.com/exdb/mnist/&gt;`_ Dataset.\n",
              "\n",
              "Args:\n",
              "    root (str or ``pathlib.Path``): Root directory of dataset where ``MNIST/raw/train-images-idx3-ubyte``\n",
              "        and  ``MNIST/raw/t10k-images-idx3-ubyte`` exist.\n",
              "    train (bool, optional): If True, creates dataset from ``train-images-idx3-ubyte``,\n",
              "        otherwise from ``t10k-images-idx3-ubyte``.\n",
              "    download (bool, optional): If True, downloads the dataset from the internet and\n",
              "        puts it in root directory. If dataset is already downloaded, it is not\n",
              "        downloaded again.\n",
              "    transform (callable, optional): A function/transform that  takes in a PIL image\n",
              "        and returns a transformed version. E.g, ``transforms.RandomCrop``\n",
              "    target_transform (callable, optional): A function/transform that takes in the\n",
              "        target and transforms it.</pre>\n",
              "      <script>\n",
              "      if (google.colab.kernel.accessAllowed && google.colab.files && google.colab.files.view) {\n",
              "        for (const element of document.querySelectorAll('.filepath')) {\n",
              "          element.style.display = 'block'\n",
              "          element.onclick = (event) => {\n",
              "            event.preventDefault();\n",
              "            event.stopPropagation();\n",
              "            google.colab.files.view(element.textContent, 20);\n",
              "          };\n",
              "        }\n",
              "      }\n",
              "      for (const element of document.querySelectorAll('.function-repr-contents')) {\n",
              "        element.onclick = (event) => {\n",
              "          event.preventDefault();\n",
              "          event.stopPropagation();\n",
              "          element.classList.toggle('function-repr-contents-collapsed');\n",
              "        };\n",
              "      }\n",
              "      </script>\n",
              "      </div>"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#printing the shape of the data.\n",
        "\n",
        "#60,000 labled training exampls of 28x28 images.\n",
        "print(train_data.data.shape)\n",
        "\n",
        "#60,000 labels, each one corresponding to an image.\n",
        "print(train_data.targets.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CFeaWxlC7VQD",
        "outputId": "cb7b35c8-0335-4a36-a54c-258d0d80b9b7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([60000, 28, 28])\n",
            "torch.Size([60000])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#rendering a particular example\n",
        "import matplotlib.pyplot as plt\n",
        "i = 10\n",
        "plt.imshow(train_data.data[i], cmap='gray')\n",
        "print(train_data.targets[i])\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 448
        },
        "id": "xbhuaVWT7Yen",
        "outputId": "081cb129-259a-466f-95df-393f43826544"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor(3)\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbT0lEQVR4nO3df2xV9f3H8dflRy8gvbcrtb2t/LCAihOoG4OuUZlKR9ttRJQt4PwDFwPDFTNBZek2QTeTTjYdYWO6PwzMTPBHNmCahaiVlmwrOBBCiNrQWm0ZtExM74UihbSf7x98veNKC57LvX3fW56P5CT03vPpeXO89Onpvb31OeecAADoZ4OsBwAAXJ4IEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMDHEeoDP6+np0eHDh5WZmSmfz2c9DgDAI+ecjh8/roKCAg0a1Pd1TsoF6PDhwxozZoz1GACAS9Ta2qrRo0f3eX/KfQsuMzPTegQAQAJc7Ot50gK0bt06XX311Ro2bJiKi4v19ttvf6F1fNsNAAaGi309T0qAXnrpJS1fvlyrVq3SO++8o6KiIpWVleno0aPJOBwAIB25JJgxY4arrKyMftzd3e0KCgpcdXX1RdeGw2EniY2NjY0tzbdwOHzBr/cJvwI6ffq09uzZo9LS0uhtgwYNUmlpqerr68/bv6urS5FIJGYDAAx8CQ/Qxx9/rO7ubuXl5cXcnpeXp7a2tvP2r66uVjAYjG68Ag4ALg/mr4KrqqpSOByObq2trdYjAQD6QcJ/DignJ0eDBw9We3t7zO3t7e0KhULn7e/3++X3+xM9BgAgxSX8CigjI0PTpk1TTU1N9Laenh7V1NSopKQk0YcDAKSppLwTwvLly7Vw4UJ97Wtf04wZM7RmzRp1dnbqBz/4QTIOBwBIQ0kJ0Pz58/Xf//5XK1euVFtbm2688UZt27btvBcmAAAuXz7nnLMe4lyRSETBYNB6DADAJQqHwwoEAn3eb/4qOADA5YkAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYGGI9AJAMX/7yl+Na953vfMfzmsWLF3te8+9//9vzmr1793peE681a9Z4XnP69OnED4IBjSsgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMCEzznnrIc4VyQSUTAYtB4DKeSHP/yh5zW/+c1v4jrWyJEj41o30Nx+++2e12zfvj0JkyCdhcNhBQKBPu/nCggAYIIAAQBMJDxAjz32mHw+X8w2adKkRB8GAJDmkvIL6W644Qa9+eab/zvIEH7vHQAgVlLKMGTIEIVCoWR8agDAAJGU54AOHjyogoICjR8/Xvfcc49aWlr63Lerq0uRSCRmAwAMfAkPUHFxsTZs2KBt27bpmWeeUXNzs2655RYdP3681/2rq6sVDAaj25gxYxI9EgAgBSU8QBUVFfre976nqVOnqqysTH//+9/V0dGhl19+udf9q6qqFA6Ho1tra2uiRwIApKCkvzogKytL1157rRobG3u93+/3y+/3J3sMAECKSfrPAZ04cUJNTU3Kz89P9qEAAGkk4QF6+OGHVVdXpw8//FD/+te/dOedd2rw4MG6++67E30oAEAaS/i34A4dOqS7775bx44d05VXXqmbb75ZO3fu1JVXXpnoQwEA0hhvRoqUl52d7XnNe++9F9excnNz41o30HR0dHheM3/+fM9rXn/9dc9rkD54M1IAQEoiQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwk/RfSAZfqk08+8bxm1apVcR3rqaee8rxmxIgRnte0tLR4XjN27FjPa+KVlZXleU15ebnnNbwZ6eWNKyAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY8DnnnPUQ54pEIgoGg9Zj4DK1b98+z2uKioo8rzlw4IDnNZMnT/a8pj9NmDDB85oPPvggCZMgVYTDYQUCgT7v5woIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADAxxHoAIJU88cQTntf87Gc/87zmxhtv9Lwm1WVkZFiPgDTDFRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYMLnnHPWQ5wrEokoGAxajwF8YaFQyPOa119/3fOaKVOmeF7Tn/7yl794XvPd7343CZMgVYTDYQUCgT7v5woIAGCCAAEATHgO0I4dOzRnzhwVFBTI5/Npy5YtMfc757Ry5Url5+dr+PDhKi0t1cGDBxM1LwBggPAcoM7OThUVFWndunW93r969WqtXbtWzz77rHbt2qUrrrhCZWVlOnXq1CUPCwAYODz/RtSKigpVVFT0ep9zTmvWrNHPf/5z3XHHHZKk559/Xnl5edqyZYsWLFhwadMCAAaMhD4H1NzcrLa2NpWWlkZvCwaDKi4uVn19fa9rurq6FIlEYjYAwMCX0AC1tbVJkvLy8mJuz8vLi973edXV1QoGg9FtzJgxiRwJAJCizF8FV1VVpXA4HN1aW1utRwIA9IOEBuizH8hrb2+Pub29vb3PH9bz+/0KBAIxGwBg4EtogAoLCxUKhVRTUxO9LRKJaNeuXSopKUnkoQAAac7zq+BOnDihxsbG6MfNzc3at2+fsrOzNXbsWD344IN64okndM0116iwsFCPPvqoCgoKNHfu3ETODQBIc54DtHv3bt12223Rj5cvXy5JWrhwoTZs2KAVK1aos7NTixcvVkdHh26++WZt27ZNw4YNS9zUAIC0x5uRAue45557PK8pKiryvObhhx/2vMbn83le05+WLVvmec2aNWsSPwhSBm9GCgBISQQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDh+dcxAP1t0qRJntds3rw5rmNNnDjR85ohQ/hnJEl/+9vfrEdAmuEKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwwbsoIuVdf/31ntcUFhbGdSzeWDR+y5Yt87zmgQceSMIkSBdcAQEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJnjnRaS8zZs3e16zYsWKuI715JNPel4zbNiwuI410OTn51uPgDTDFRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYII3I8WAtHbt2rjWHTx40POarKysuI7l1ZAh3v+5/v73v4/rWIFAIK51gBdcAQEATBAgAIAJzwHasWOH5syZo4KCAvl8Pm3ZsiXm/nvvvVc+ny9mKy8vT9S8AIABwnOAOjs7VVRUpHXr1vW5T3l5uY4cORLdNm3adElDAgAGHs/PalZUVKiiouKC+/j9foVCobiHAgAMfEl5Dqi2tla5ubm67rrrdP/99+vYsWN97tvV1aVIJBKzAQAGvoQHqLy8XM8//7xqamr05JNPqq6uThUVFeru7u51/+rqagWDweg2ZsyYRI8EAEhBCf85oAULFkT/PGXKFE2dOlUTJkxQbW2tZs2add7+VVVVWr58efTjSCRChADgMpD0l2GPHz9eOTk5amxs7PV+v9+vQCAQswEABr6kB+jQoUM6duyY8vPzk30oAEAa8fwtuBMnTsRczTQ3N2vfvn3Kzs5Wdna2Hn/8cc2bN0+hUEhNTU1asWKFJk6cqLKysoQODgBIb54DtHv3bt12223Rjz97/mbhwoV65plntH//fv3pT39SR0eHCgoKNHv2bP3yl7+U3+9P3NQAgLTnc8456yHOFYlEFAwGrccAUo7P5/O85rHHHovrWCtXrvS8pqmpyfOa3l6YdDEfffSR5zWwEQ6HL/i8Pu8FBwAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMJ/5XcAJIjIyPD85p43tU6XmfOnPG8pru7OwmTIF1wBQQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmODNSIE08cQTT1iPcEHPPfec5zWHDh1KwiRIF1wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmfM45Zz3EuSKRiILBoPUYaWvUqFGe16xfvz6uY23atKlf1gxE+fn5nte8//77ntcEAgHPa+I1YcIEz2s++OCDJEyCVBEOhy/4GOQKCABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwMcR6ACTW2rVrPa+ZM2dOXMe69tprPa85fPiw5zX/+c9/PK9pbGz0vEaSpk2b5nlNPOdhxYoVntf05xuLPvXUU57XxPPfFpc3roAAACYIEADAhKcAVVdXa/r06crMzFRubq7mzp2rhoaGmH1OnTqlyspKjRo1SiNHjtS8efPU3t6e0KEBAOnPU4Dq6upUWVmpnTt36o033tCZM2c0e/ZsdXZ2RvdZtmyZXn31Vb3yyiuqq6vT4cOHdddddyV8cABAevP0IoRt27bFfLxhwwbl5uZqz549mjlzpsLhsJ577jlt3LhRt99+u6Szv23z+uuv186dO/X1r389cZMDANLaJT0HFA6HJUnZ2dmSpD179ujMmTMqLS2N7jNp0iSNHTtW9fX1vX6Orq4uRSKRmA0AMPDFHaCenh49+OCDuummmzR58mRJUltbmzIyMpSVlRWzb15entra2nr9PNXV1QoGg9FtzJgx8Y4EAEgjcQeosrJSBw4c0IsvvnhJA1RVVSkcDke31tbWS/p8AID0ENcPoi5dulSvvfaaduzYodGjR0dvD4VCOn36tDo6OmKugtrb2xUKhXr9XH6/X36/P54xAABpzNMVkHNOS5cu1ebNm/XWW2+psLAw5v5p06Zp6NChqqmpid7W0NCglpYWlZSUJGZiAMCA4OkKqLKyUhs3btTWrVuVmZkZfV4nGAxq+PDhCgaDuu+++7R8+XJlZ2crEAjogQceUElJCa+AAwDE8BSgZ555RpJ06623xty+fv163XvvvZKk3/72txo0aJDmzZunrq4ulZWV6Q9/+ENChgUADBw+55yzHuJckUhEwWDQeoy0Fc+V5tNPPx3Xsfrr26offvih5zXvvvtuXMe65ZZbPK/JzMyM61hexfNP9f3334/rWNOnT/e85twfSAeksz+qc6E30eW94AAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCd8OGnnrqqbjWNTY2el7Dr+aI3yeffOJ5zahRo5IwCfDF8G7YAICURIAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYGGI9AOw99NBDca3z+/2e14wcOTKuY3n1la98Ja51d999d4In6V04HPa85pvf/GYSJgHscAUEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjwOeec9RDnikQiCgaD1mMAAC5ROBxWIBDo836ugAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJTwGqrq7W9OnTlZmZqdzcXM2dO1cNDQ0x+9x6663y+Xwx25IlSxI6NAAg/XkKUF1dnSorK7Vz50698cYbOnPmjGbPnq3Ozs6Y/RYtWqQjR45Et9WrVyd0aABA+hviZedt27bFfLxhwwbl5uZqz549mjlzZvT2ESNGKBQKJWZCAMCAdEnPAYXDYUlSdnZ2zO0vvPCCcnJyNHnyZFVVVenkyZN9fo6uri5FIpGYDQBwGXBx6u7udt/+9rfdTTfdFHP7H//4R7dt2za3f/9+9+c//9ldddVV7s477+zz86xatcpJYmNjY2MbYFs4HL5gR+IO0JIlS9y4ceNca2vrBferqalxklxjY2Ov9586dcqFw+Ho1traan7S2NjY2NgufbtYgDw9B/SZpUuX6rXXXtOOHTs0evToC+5bXFwsSWpsbNSECRPOu9/v98vv98czBgAgjXkKkHNODzzwgDZv3qza2loVFhZedM2+ffskSfn5+XENCAAYmDwFqLKyUhs3btTWrVuVmZmptrY2SVIwGNTw4cPV1NSkjRs36lvf+pZGjRql/fv3a9myZZo5c6amTp2alL8AACBNeXneR318n2/9+vXOOedaWlrczJkzXXZ2tvP7/W7ixInukUceuej3Ac8VDofNv2/JxsbGxnbp28W+9vv+PywpIxKJKBgMWo8BALhE4XBYgUCgz/t5LzgAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgImUC5BzznoEAEACXOzrecoF6Pjx49YjAAAS4GJfz30uxS45enp6dPjwYWVmZsrn88XcF4lENGbMGLW2tioQCBhNaI/zcBbn4SzOw1mch7NS4Tw453T8+HEVFBRo0KC+r3OG9ONMX8igQYM0evToC+4TCAQu6wfYZzgPZ3EezuI8nMV5OMv6PASDwYvuk3LfggMAXB4IEADARFoFyO/3a9WqVfL7/dajmOI8nMV5OIvzcBbn4ax0Og8p9yIEAMDlIa2ugAAAAwcBAgCYIEAAABMECABgIm0CtG7dOl199dUaNmyYiouL9fbbb1uP1O8ee+wx+Xy+mG3SpEnWYyXdjh07NGfOHBUUFMjn82nLli0x9zvntHLlSuXn52v48OEqLS3VwYMHbYZNooudh3vvvfe8x0d5ebnNsElSXV2t6dOnKzMzU7m5uZo7d64aGhpi9jl16pQqKys1atQojRw5UvPmzVN7e7vRxMnxRc7Drbfeet7jYcmSJUYT9y4tAvTSSy9p+fLlWrVqld555x0VFRWprKxMR48etR6t391www06cuRIdPvHP/5hPVLSdXZ2qqioSOvWrev1/tWrV2vt2rV69tlntWvXLl1xxRUqKyvTqVOn+nnS5LrYeZCk8vLymMfHpk2b+nHC5Kurq1NlZaV27typN954Q2fOnNHs2bPV2dkZ3WfZsmV69dVX9corr6iurk6HDx/WXXfdZTh14n2R8yBJixYtink8rF692mjiPrg0MGPGDFdZWRn9uLu72xUUFLjq6mrDqfrfqlWrXFFRkfUYpiS5zZs3Rz/u6elxoVDI/frXv47e1tHR4fx+v9u0aZPBhP3j8+fBOecWLlzo7rjjDpN5rBw9etRJcnV1dc65s//thw4d6l555ZXoPu+9956T5Orr663GTLrPnwfnnPvGN77hfvzjH9sN9QWk/BXQ6dOntWfPHpWWlkZvGzRokEpLS1VfX284mY2DBw+qoKBA48eP1z333KOWlhbrkUw1Nzerra0t5vERDAZVXFx8WT4+amtrlZubq+uuu07333+/jh07Zj1SUoXDYUlSdna2JGnPnj06c+ZMzONh0qRJGjt27IB+PHz+PHzmhRdeUE5OjiZPnqyqqiqdPHnSYrw+pdybkX7exx9/rO7ubuXl5cXcnpeXp/fff99oKhvFxcXasGGDrrvuOh05ckSPP/64brnlFh04cECZmZnW45loa2uTpF4fH5/dd7koLy/XXXfdpcLCQjU1NemnP/2pKioqVF9fr8GDB1uPl3A9PT168MEHddNNN2ny5MmSzj4eMjIylJWVFbPvQH489HYeJOn73/++xo0bp4KCAu3fv18/+clP1NDQoL/+9a+G08ZK+QDhfyoqKqJ/njp1qoqLizVu3Di9/PLLuu+++wwnQypYsGBB9M9TpkzR1KlTNWHCBNXW1mrWrFmGkyVHZWWlDhw4cFk8D3ohfZ2HxYsXR/88ZcoU5efna9asWWpqatKECRP6e8xepfy34HJycjR48ODzXsXS3t6uUChkNFVqyMrK0rXXXqvGxkbrUcx89hjg8XG+8ePHKycnZ0A+PpYuXarXXntN27dvj/n1LaFQSKdPn1ZHR0fM/gP18dDXeehNcXGxJKXU4yHlA5SRkaFp06appqYmeltPT49qampUUlJiOJm9EydOqKmpSfn5+dajmCksLFQoFIp5fEQiEe3ateuyf3wcOnRIx44dG1CPD+ecli5dqs2bN+utt95SYWFhzP3Tpk3T0KFDYx4PDQ0NamlpGVCPh4udh97s27dPklLr8WD9Kogv4sUXX3R+v99t2LDBvfvuu27x4sUuKyvLtbW1WY/Wrx566CFXW1vrmpub3T//+U9XWlrqcnJy3NGjR61HS6rjx4+7vXv3ur179zpJ7umnn3Z79+51H330kXPOuV/96lcuKyvLbd261e3fv9/dcccdrrCw0H366afGkyfWhc7D8ePH3cMPP+zq6+tdc3Oze/PNN91Xv/pVd80117hTp05Zj54w999/vwsGg662ttYdOXIkup08eTK6z5IlS9zYsWPdW2+95Xbv3u1KSkpcSUmJ4dSJd7Hz0NjY6H7xi1+43bt3u+bmZrd161Y3fvx4N3PmTOPJY6VFgJxz7ne/+50bO3asy8jIcDNmzHA7d+60HqnfzZ8/3+Xn57uMjAx31VVXufnz57vGxkbrsZJu+/btTtJ528KFC51zZ1+K/eijj7q8vDzn9/vdrFmzXENDg+3QSXCh83Dy5Ek3e/Zsd+WVV7qhQ4e6cePGuUWLFg24/0nr7e8vya1fvz66z6effup+9KMfuS996UtuxIgR7s4773RHjhyxGzoJLnYeWlpa3MyZM112drbz+/1u4sSJ7pFHHnHhcNh28M/h1zEAAEyk/HNAAICBiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAw8X8Qb6lOzQWODQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "torch.set_printoptions(linewidth=200)\n",
        "i = 10\n",
        "print(train_data.data[i])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jpJRcWEwG8zF",
        "outputId": "d701d3fe-3a8c-4ef7-bfb7-0f032b3b08bb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  42, 118, 219, 166, 118, 118,   6,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 103, 242, 254, 254, 254, 254, 254,  66,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  18, 232, 254, 254, 254, 254, 254, 238,  70,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 104, 244, 254, 224, 254, 254, 254, 141,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 207, 254, 210, 254, 254, 254,  34,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  84, 206, 254, 254, 254, 254,  41,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  24, 209, 254, 254, 254, 171,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  91, 137, 253, 254, 254, 254, 112,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  40, 214, 250, 254, 254, 254, 254, 254,  34,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  81, 247, 254, 254, 254, 254, 254, 254, 146,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 110, 246, 254, 254, 254, 254, 254, 171,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  73,  89,  89,  93, 240, 254, 171,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1, 128, 254, 219,  31,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   7, 254, 254, 214,  28,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 138, 254, 254, 116,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,  19, 177,  90,   0,   0,   0,   0,   0,  25, 240, 254, 254,  34,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0, 164, 254, 215,  63,  36,   0,  51,  89, 206, 254, 254, 139,   8,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,  57, 197, 254, 254, 222, 180, 241, 254, 254, 253, 213,  11,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0, 140, 105, 254, 254, 254, 254, 254, 254, 236,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   7, 117, 117, 165, 254, 254, 239,  50,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0],\n",
            "        [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=torch.uint8)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Defining the Model\n",
        "this is kind of clever, I found this while Googling.\n",
        "instead of needing to call each layer, like I did\n",
        "```\n",
        "class network(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(network, self).__init__()\n",
        "        self.dense1 = nn.Linear(784,10)\n",
        "        self.dense2 = nn.Linear(10,10)\n",
        "        ...\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = torch.flatten(x)\n",
        "        x = torch.relu(self.dense1(x))\n",
        "        x = torch.relu(self.dense2(x))\n",
        "        ...\n",
        "        return x\n",
        "```\n",
        "\n",
        "you can use `nn.Sequential` to wrap everything up into a sequence that just needs to be called once. Saves on some code, and easier to develop on.\n",
        "\n",
        "Also, based on this article:\n",
        "\n",
        "https://medium.com/@ramamurthi96/a-simple-neural-network-model-for-mnist-using-pytorch-4b8b148ecbdc\n",
        "\n",
        "apparently I didn't need to put softmax at the end of the model. Maybe that's what was going wrong."
      ],
      "metadata": {
        "id": "dYTTR2K97mUT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torch import nn\n",
        "import torch\n",
        "\n",
        "class MNISTModel(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(MNISTModel, self).__init__()\n",
        "        self.fc1 = nn.Linear(784, 128)\n",
        "        self.activation1 = nn.ReLU()\n",
        "        self.fc2 = nn.Linear(128, 64)\n",
        "        self.activation2 = nn.ReLU()\n",
        "        self.fc3 = nn.Linear(64, 10)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = torch.flatten(x)\n",
        "        x = self.fc1(x)\n",
        "        x = self.activation1(x)\n",
        "        x = self.fc2(x)\n",
        "        x = self.activation2(x)\n",
        "        x = self.fc3(x)\n",
        "        return x\n",
        "\n",
        "#defining the model and printing it.\n",
        "#it gets re-defined later just so I don't accidentally train the\n",
        "#same model a few times.\n",
        "model = MNISTModel()\n",
        "print(model)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xTx-i4Y37hf0",
        "outputId": "5eb995d7-5003-4944-d9ac-48650700abda"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MNISTModel(\n",
            "  (fc1): Linear(in_features=784, out_features=128, bias=True)\n",
            "  (activation1): ReLU()\n",
            "  (fc2): Linear(in_features=128, out_features=64, bias=True)\n",
            "  (activation2): ReLU()\n",
            "  (fc3): Linear(in_features=64, out_features=10, bias=True)\n",
            ")\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Training"
      ],
      "metadata": {
        "id": "_fctADgE8oxq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "from tqdm import tqdm\n",
        "\n",
        "model = MNISTModel()\n",
        "loss_fn = nn.CrossEntropyLoss()\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=1e-4)\n",
        "\n",
        "losses = []\n",
        "\n",
        "for i, (x, y) in tqdm(enumerate(zip(train_data.data, train_data.targets))):\n",
        "\n",
        "    #putting the input to the model onto the device\n",
        "    x = x.type(torch.float32)\n",
        "\n",
        "    #having\n",
        "    y_pred = model.forward(x)\n",
        "\n",
        "    #re-formatting the output to instead of 0-10 it's [1,0,0,0,0,0,0,0,0,0] - [0,0,0,0,0,0,0,0,0,1]\n",
        "    y = F.one_hot(y,10).type(torch.float32)\n",
        "\n",
        "    loss = loss_fn(y_pred, y)\n",
        "\n",
        "    # Backward pass (updating model based on gradients)\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    #keeping track of loss\n",
        "    losses.append(float(loss))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 543
        },
        "id": "BmcSJz5v8adz",
        "outputId": "e1193169-37b6-49e0-cea3-3986bbc84069"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "60000it [01:28, 681.11it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "60000it [01:29, 673.74it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "60000it [01:28, 680.39it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "60000it [01:29, 671.06it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "3\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "15850it [00:23, 663.92it/s]\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-34-0d9c71abd9c4>\u001b[0m in \u001b[0;36m<cell line: 11>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     26\u001b[0m         \u001b[0moptimizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzero_grad\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m         \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 28\u001b[0;31m         \u001b[0moptimizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     29\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     30\u001b[0m         \u001b[0;31m#keeping track of loss\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    389\u001b[0m                             )\n\u001b[1;32m    390\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 391\u001b[0;31m                 \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    392\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_optimizer_step_code\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    393\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py\u001b[0m in \u001b[0;36m_use_grad\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     74\u001b[0m             \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_grad_enabled\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdefaults\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'differentiable'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     75\u001b[0m             \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_dynamo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph_break\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 76\u001b[0;31m             \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     77\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     78\u001b[0m             \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_dynamo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph_break\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/optim/sgd.py\u001b[0m in \u001b[0;36mstep\u001b[0;34m(self, closure)\u001b[0m\n\u001b[1;32m     78\u001b[0m             \u001b[0mhas_sparse_grad\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_init_group\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams_with_grad\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0md_p_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmomentum_buffer_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     79\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 80\u001b[0;31m             sgd(params_with_grad,\n\u001b[0m\u001b[1;32m     81\u001b[0m                 \u001b[0md_p_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     82\u001b[0m                 \u001b[0mmomentum_buffer_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/optim/sgd.py\u001b[0m in \u001b[0;36msgd\u001b[0;34m(params, d_p_list, momentum_buffer_list, has_sparse_grad, foreach, fused, grad_scale, found_inf, weight_decay, momentum, lr, dampening, nesterov, maximize)\u001b[0m\n\u001b[1;32m    243\u001b[0m         \u001b[0mfunc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_single_tensor_sgd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    244\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 245\u001b[0;31m     func(params,\n\u001b[0m\u001b[1;32m    246\u001b[0m          \u001b[0md_p_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m          \u001b[0mmomentum_buffer_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/optim/sgd.py\u001b[0m in \u001b[0;36m_single_tensor_sgd\u001b[0;34m(params, d_p_list, momentum_buffer_list, grad_scale, found_inf, weight_decay, momentum, lr, dampening, nesterov, maximize, has_sparse_grad)\u001b[0m\n\u001b[1;32m    291\u001b[0m                 \u001b[0md_p\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbuf\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    292\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 293\u001b[0;31m         \u001b[0mparam\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd_\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md_p\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mlr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    294\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    295\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "#if we just plot the noise of individual samples we'll get a very sporadic line'\n",
        "#this smooths out the trend of our data.\n",
        "def moving_avg(x, n):\n",
        "    cumsum = np.cumsum(np.insert(x, 0, 0))\n",
        "    return (cumsum[n:] - cumsum[:-n]) / float(n)\n",
        "\n",
        "plt.plot(moving_avg(losses, 1000))\n",
        "plt.xlabel('training iteration')\n",
        "plt.ylabel('loss')\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "id": "8CgPvCFx-OwV",
        "outputId": "fb5969eb-0658-4b4a-bd76-eed36c8e6539"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfA0lEQVR4nO3deVxUVf8H8M8MywAiICKrKLjvghvhbqK4ZNrqoz25VPqrbJO0pFyyRczUbLEsTa2eyjYzC3eUXHIX3DcUBBVwZ1W2ub8/YC5zZ2PAGS4Mn/frxeuZuXPunTMXHufbOd/zPQpBEAQQERER2Qil3B0gIiIisiQGN0RERGRTGNwQERGRTWFwQ0RERDaFwQ0RERHZFAY3REREZFMY3BAREZFNsZe7A9VNrVbj6tWrqF+/PhQKhdzdISIiIjMIgoCcnBz4+/tDqTQ9NlPngpurV68iMDBQ7m4QERFRFaSlpaFx48Ym29S54KZ+/foASm+Om5ubzL0hIiIic2RnZyMwMFD8HjelzgU3mqkoNzc3BjdERES1jDkpJUwoJiIiIpvC4IaIiIhsCoMbIiIisikMboiIiMimMLghIiIim8LghoiIiGwKgxsiIiKyKQxuiIiIyKbIGtzs3LkTI0aMgL+/PxQKBdatW1fhOQUFBXj77bfRtGlTqFQqBAUFYeXKldbvLBEREdUKslYozsvLQ+fOnfHMM8/g0UcfNeucJ598EpmZmfjmm2/QokULpKenQ61WW7mnREREVFvIGtwMHToUQ4cONbv9pk2b8M8//+DixYvw9PQEAAQFBVmpd0RERFQb1aqcm/Xr16Nbt25YsGABAgIC0KpVK0ybNg137941ek5BQQGys7MlP0RERGS7atXGmRcvXsTu3bvh5OSEP/74Azdu3MCLL76ImzdvYtWqVQbPiYmJwdy5c63etxK1gPSs0iCrcQMXq78fERERGVarRm7UajUUCgV++OEH9OjRA8OGDcPixYvx7bffGh29iY6ORlZWlviTlpZmlb7dzCtA7w93oO+CHVa5PhEREZmnVo3c+Pn5ISAgAO7u7uKxtm3bQhAEXL58GS1bttQ7R6VSQaVSVWc3iYiISEa1auSmV69euHr1KnJzc8Vj586dg1KpROPGjWXsGREREdUUsgY3ubm5SExMRGJiIgAgOTkZiYmJSE1NBVA6pTRu3Dix/dixY9GwYUNMnDgRp06dws6dOzF9+nQ888wzcHZ2luMjEBERUQ0ja3Bz6NAhhIaGIjQ0FAAQFRWF0NBQzJ49GwCQnp4uBjoA4Orqiq1bt+LOnTvo1q0bnnrqKYwYMQKffvqpLP03RJC7A0RERHWcrDk3/fv3hyAYDwdWr16td6xNmzbYunWrFXtFREREtVmtyrkhIiIiqgiDGwtRQCF3F4iIiAgMbizOxCwbERERVQMGN0RERGRTGNwQERGRTWFwQ0RERDaFwQ0RERHZFAY3REREZFMY3FiIQmsluKnChERERGRdDG4shFVuiIiIagYGN1bAgRsiIiL5MLixEIXWvBRjGyIiIvkwuLEQTksRERHVDAxurIAJxURERPJhcGMhCg7dEBER1QgMbixEe1dwjtsQERHJh8GNFXBWioiISD4MbiyF01JEREQ1AoMbC5FUKObEFBERkWwY3FgBp6WIiIjkw+DGQjgrRUREVDMwuCEiIiKbwuDGQhQsdENERFQjMLixEO3Qhjk3RERE8mFwYwVcLUVERCQfBjcWwlkpIiKimoHBjYVItl/gwA0REZFsGNxYAWMbIiIi+TC4sRBOSxEREdUMDG6sQOC8FBERkWwY3FgBQxsiIiL5MLixEE5LERER1QyyBjc7d+7EiBEj4O/vD4VCgXXr1pl97p49e2Bvb4+QkBCr9a8yuFqKiIioZpA1uMnLy0Pnzp2xdOnSSp13584djBs3DgMHDrRSz4iIiKi2spfzzYcOHYqhQ4dW+rznn38eY8eOhZ2dXYWjPQUFBSgoKBCfZ2dnV/r9zCGZluLIDRERkWxqXc7NqlWrcPHiRcyZM8es9jExMXB3dxd/AgMDrdIvaWzD6IaIiEgutSq4OX/+PGbMmIH//e9/sLc3b9ApOjoaWVlZ4k9aWpqVe0lERERyknVaqjJKSkowduxYzJ07F61atTL7PJVKBZVKZcWelVIomFBMRERUE9Sa4CYnJweHDh1CQkICXnrpJQCAWq2GIAiwt7fHli1b8OCDD8rcy1KMbYiIiORTa4IbNzc3HD9+XHLsiy++wPbt2/Hbb78hODhYpp6VYpkbIiKimkHW4CY3NxdJSUni8+TkZCQmJsLT0xNNmjRBdHQ0rly5gu+++w5KpRIdOnSQnO/t7Q0nJye943LQXi3F7ReIiIjkI2twc+jQIQwYMEB8HhUVBQAYP348Vq9ejfT0dKSmpsrVPSIiIqqFFEIdG2bIzs6Gu7s7srKy4ObmZtFrB82IBQAcmhkBL1frJzETERHVFZX5/q5VS8Fri7oVLhIREdUsDG6IiIjIpjC4sSBNUjErFBMREcmHwY0FiQumGNsQERHJhsENERER2RQGNxak2YKBAzdERETyYXBjQaxSTEREJD8GN1bApeBERETyYXBjQVwtRUREJD8GN0RERGRTGNxYkKIs64bTUkRERPJhcGNJ4rQUERERyYXBDREREdkUBjcWpFkKXsc2WiciIqpRGNxYkIKFboiIiGTH4MYKOHBDREQkHwY3FqRgjWIiIiLZMbixoLtFJXJ3gYiIqM5jcGMFuQXFcneBiIiozmJwYwVODnZyd4GIiKjOYnBjQfWd7AFwKTgREZGcGNxYkLJsLbiasQ0REZFsGNxYkFKz/QJHboiIiGTD4MaCOHJDREQkPwY3FqQQgxtGN0RERHJhcGNB5dNS8vaDiIioLmNwY0GavaU4ckNERCQfBjcWpMm5YWxDREQkHwY3FqRkzg0REZHsGNxYkGZaiqENERGRfBjcWBBHboiIiOQna3Czc+dOjBgxAv7+/lAoFFi3bp3J9mvXrsWgQYPQqFEjuLm5ITw8HJs3b66ezppBwSJ+REREspM1uMnLy0Pnzp2xdOlSs9rv3LkTgwYNwoYNG3D48GEMGDAAI0aMQEJCgpV7ap5LN/MBAMUlDG6IiIjkYi/nmw8dOhRDhw41u/2SJUskz+fNm4c///wTf/31F0JDQy3cu6r7I+EKwpo1lLsbREREdZKswc39UqvVyMnJgaenp9E2BQUFKCgoEJ9nZ2dbvV83cgut/h5ERERkWK1OKF64cCFyc3Px5JNPGm0TExMDd3d38ScwMNDq/XqiW2OrvwcREREZVmuDmx9//BFz587FL7/8Am9vb6PtoqOjkZWVJf6kpaVZrU8hgR4AADtNZjERERFVu1o5LbVmzRo899xz+PXXXxEREWGyrUqlgkqlqpZ+2ZdtLlXMbcGJiIhkU+tGbn766SdMnDgRP/30E4YPHy53dyTsyoKbEgY3REREspF15CY3NxdJSUni8+TkZCQmJsLT0xNNmjRBdHQ0rly5gu+++w5A6VTU+PHj8cknnyAsLAwZGRkAAGdnZ7i7u8vyGbTZ22lGbtQy94SIiKjuknXk5tChQwgNDRWXcUdFRSE0NBSzZ88GAKSnpyM1NVVs//XXX6O4uBhTpkyBn5+f+PPqq6/K0n9ddsrS28mRGyIiIvnIOnLTv39/k9V8V69eLXkeHx9v3Q7dp7KBG+bcEBERyajW5dzUZBy5ISIikh+DGwuyZ0IxERGR7BjcWJCdHYMbIiIiuTG4sSDWuSEiIpIfgxsLKq9zw6XgREREcmFwY0GabRc4ckNERCQfBjcWpCniV1LC4IaIiEguDG4sSJyWMlG7h4iIiKyLwY0F2bPODRERkewY3FiQHVdLERERyY7BjQWxiB8REZH8GNxYkDhyw4RiIiIi2TC4sSDWuSEiIpIfgxsLYs4NERGR/BjcWJAm50bNpeBERESyYXBjQfmFJQCAfy/clLknREREdReDGwv6Iv4CAODSzXyZe0JERFR3MbixoEb1VXJ3gYiIqM5jcGNBrwxsKXcXiIiI6jwGNxb0v72X5O4CERFRncfgxoIeDvGXuwtERER1HoMbC2rrVx8A0M7PTeaeEBER1V0MbixIoVCU/a/MHSEiIqrDGNxYkJ1CU8RP5o4QERHVYQxuLEipCW4Y3RAREcmGwY0FKcvuJrdfICIikg+DGwvSjNyUMLghIiKSDYMbC9LsCs5pKSIiIvkwuLEgJROKiYiIZMfgxoLKBm5QwuiGiIhINgxuLEiclmLODRERkWwY3FhQ+bQUgxsiIiK5yBrc7Ny5EyNGjIC/vz8UCgXWrVtX4Tnx8fHo0qULVCoVWrRogdWrV1u9n+YSV0upZe4IERFRHSZrcJOXl4fOnTtj6dKlZrVPTk7G8OHDMWDAACQmJuK1117Dc889h82bN1u5p+bRTEsJHLkhIiKSjb2cbz506FAMHTrU7PbLli1DcHAwFi1aBABo27Ytdu/ejY8//hiRkZEGzykoKEBBQYH4PDs7+/46bYJdWajIOjdERETyqVU5N3v37kVERITkWGRkJPbu3Wv0nJiYGLi7u4s/gYGBVuufQpyWYnBDREQkl1oV3GRkZMDHx0dyzMfHB9nZ2bh7967Bc6Kjo5GVlSX+pKWlWa1/mo0zOXBDREQkH1mnpaqDSqWCSqWqlvdScuSGiIhIdrVq5MbX1xeZmZmSY5mZmXBzc4Ozs7NMvSqnZM4NERGR7GpVcBMeHo64uDjJsa1btyI8PFymHklxtRQREZH8ZA1ucnNzkZiYiMTERAClS70TExORmpoKoDRfZty4cWL7559/HhcvXsQbb7yBM2fO4IsvvsAvv/yCqVOnytF9PZyWIiIikp+swc2hQ4cQGhqK0NBQAEBUVBRCQ0Mxe/ZsAEB6eroY6ABAcHAwYmNjsXXrVnTu3BmLFi3CihUrjC4Dr27cOJOIiEh+siYU9+/f3+QUjqHqw/3790dCQoIVe1V1mmkpAFCrBSi1nhMREVH1qFU5NzWddizDpGIiIiJ5MLixIO2RGm6eSUREJA8GNxakKeIHAGpunklERCQLBjcWpNQKbjgtRUREJA8GNxak1LqbnJYiIiKSB4MbC5JOSzG4ISIikgODGwvSnpZibENERCQPBjcWpL1ailWKiYiI5MHgxsI0hfyYc0NERCQPBjcWphm8YXBDREQkDwY3FsbNM4mIiOTF4MbCxM0zWcSPiIhIFgxuLIw5N0RERPJicGNhmpwbVigmIiKSB4MbC9MsB2cRPyIiInkwuLEwTZVixjZERETyYHBjYZqRG66WIiIikgeDGwtjnRsiIiJ5MbixsPJpKQY3REREcmBwY2GcliIiIpIXgxsLUzKhmIiISFYMbiyMRfyIiIjkxeDGwsQifhy6ISIikgWDGwtTMqGYiIhIVgxuLEycluLGmURERLJgcGNhCo7cEBERyapKwc23336L2NhY8fkbb7wBDw8P9OzZE5cuXbJY52oju7I7yo0ziYiI5FGl4GbevHlwdnYGAOzduxdLly7FggUL4OXlhalTp1q0g7WNWMSPCcVERESysK/KSWlpaWjRogUAYN26dXjssccwefJk9OrVC/3797dk/2odBevcEBERyapKIzeurq64efMmAGDLli0YNGgQAMDJyQl37961XO9qITtWKCYiIpJVlUZuBg0ahOeeew6hoaE4d+4chg0bBgA4efIkgoKCLNm/Wod7SxEREcmrSiM3S5cuRXh4OK5fv47ff/8dDRs2BAAcPnwYY8aMqdL1goKC4OTkhLCwMBw4cMBk+yVLlqB169ZwdnZGYGAgpk6dinv37lXlo1icgruCExERyapKIzceHh74/PPP9Y7PnTu30tf6+eefERUVhWXLliEsLAxLlixBZGQkzp49C29vb732P/74I2bMmIGVK1eiZ8+eOHfuHCZMmACFQoHFixdX5eNYFKeliIiI5FWlkZtNmzZh9+7d4vOlS5ciJCQEY8eOxe3btyt1rcWLF2PSpEmYOHEi2rVrh2XLlsHFxQUrV6402P7ff/9Fr169MHbsWAQFBWHw4MEYM2ZMhaM91UUT3HDghoiISB5VCm6mT5+O7OxsAMDx48fx+uuvY9iwYUhOTkZUVJTZ1yksLMThw4cRERFR3iGlEhEREdi7d6/Bc3r27InDhw+LwczFixexYcMGMe9HV0FBAbKzsyU/1rTr/A0AwB8JV6z6PkRERGRYlaalkpOT0a5dOwDA77//joceegjz5s3DkSNHjAYZhty4cQMlJSXw8fGRHPfx8cGZM2cMnjN27FjcuHEDvXv3hiAIKC4uxvPPP4+33nrLYPuYmJgqTZfdr3/OXa/29yQiIqIqjtw4OjoiPz8fALBt2zYMHjwYAODp6Wn1kZH4+HjMmzcPX3zxBY4cOYK1a9ciNjYW7733nsH20dHRyMrKEn/S0tKs2j8iIiKSV5VGbnr37o2oqCj06tULBw4cwM8//wwAOHfuHBo3bmz2dby8vGBnZ4fMzEzJ8czMTPj6+ho8Z9asWXj66afx3HPPAQA6duyIvLw8TJ48GW+//TaUSmm8plKpoFKpKvPxiIiIqBar0sjN559/Dnt7e/z222/48ssvERAQAADYuHEjhgwZYvZ1HB0d0bVrV8TFxYnH1Go14uLiEB4ebvCc/Px8vQDGzs4OACAwi5eIiKjOq9LITZMmTfD333/rHf/4448rfa2oqCiMHz8e3bp1Q48ePbBkyRLk5eVh4sSJAIBx48YhICAAMTExAIARI0Zg8eLFCA0NRVhYGJKSkjBr1iyMGDFCDHKIiIio7qpScAMAJSUlWLduHU6fPg0AaN++PR5++OFKBxijR4/G9evXMXv2bGRkZCAkJASbNm0Sk4xTU1MlIzUzZ86EQqHAzJkzceXKFTRq1AgjRozABx98UNWPQkRERDZEIVRhLicpKQnDhg3DlStX0Lp1awDA2bNnERgYiNjYWDRv3tziHbWU7OxsuLu7IysrC25ubha/ftCMWPFxyvzhFr8+ERFRXVSZ7+8q5dy88soraN68OdLS0nDkyBEcOXIEqampCA4OxiuvvFKlThMRERFZQpWmpf755x/s27cPnp6e4rGGDRti/vz56NWrl8U6R0RERFRZVRq5UalUyMnJ0Tuem5sLR0fH++4UERERUVVVKbh56KGHMHnyZOzfvx+CIEAQBOzbtw/PP/88Hn74YUv3kYiIiMhsVQpuPv30UzRv3hzh4eFwcnKCk5MTevbsiRYtWmDJkiUW7iIRERGR+aqUc+Ph4YE///wTSUlJ4lLwtm3bokWLFhbtHBEREVFlmR3cVLTb944dO8THixcvrnqPiIiIiO6D2cFNQkKCWe0UCkWVO0NERER0v8wObrRHZsi4lt6uOH8tV+5uEBER1VlVSigm46IGtZK7C0RERHUagxsLq+/kAABo41tf5p4QERHVTQxuLEyzx2exutJbdhEREZEFMLixkiTm3RAREcmCwY2FHUi+JXcXiIiI6jQGNxYmcDaKiIhIVgxuLMxeyTo/REREcmJwY2EO9rylREREcuI3sYVx5IaIiEheDG4srJ2/m9xdICIiqtMY3FhYC29XubtARERUpzG4sTCl1sahahbyIyIiqnYMbixMO7jZl3xTxp4QERHVTQxuLEw7n/heUYl8HSEiIqqjGNxYWPbdYvFx2q27MvaEiIiobmJwY2Huzg7i4/xCjtwQERFVNwY3FqZyKL+lzRrVk7EnREREdRODGwtTaVUo9nJVydgTIiKiuonBjYUpFAo4O9gBALLvFsncGyIiorqHwY0V1HeyByCdoiIiIqLqwW9fK2jg4ggAUKtl7ggREVEdxODGCuzKit2UCKxQTEREVN0Y3FiBJrjh9gtERETVr0YEN0uXLkVQUBCcnJwQFhaGAwcOmGx/584dTJkyBX5+flCpVGjVqhU2bNhQTb2tmFIzcsPghoiIqNrZy92Bn3/+GVFRUVi2bBnCwsKwZMkSREZG4uzZs/D29tZrX1hYiEGDBsHb2xu//fYbAgICcOnSJXh4eFR/542wK9uCoZjBDRERUbWTPbhZvHgxJk2ahIkTJwIAli1bhtjYWKxcuRIzZszQa79y5UrcunUL//77LxwcSqsBBwUFVWeXKyROSzHnhoiIqNrJOi1VWFiIw4cPIyIiQjymVCoRERGBvXv3Gjxn/fr1CA8Px5QpU+Dj44MOHTpg3rx5KCkxvNVBQUEBsrOzJT/WdjDlNgDg4vVcq78XERERScka3Ny4cQMlJSXw8fGRHPfx8UFGRobBcy5evIjffvsNJSUl2LBhA2bNmoVFixbh/fffN9g+JiYG7u7u4k9gYKDFP4cxC7ecq7b3IiIiolI1IqG4MtRqNby9vfH111+ja9euGD16NN5++20sW7bMYPvo6GhkZWWJP2lpadXa30+2neeqKSIiomoka86Nl5cX7OzskJmZKTmemZkJX19fg+f4+fnBwcEBdnZ24rG2bdsiIyMDhYWFcHR0lLRXqVRQqeTb4+njbecQ5OWCkSEBsvWBiIioLpF15MbR0RFdu3ZFXFyceEytViMuLg7h4eEGz+nVqxeSkpKg1ir/e+7cOfj5+ekFNjXFq2sS5e4CERFRnSH7tFRUVBSWL1+Ob7/9FqdPn8YLL7yAvLw8cfXUuHHjEB0dLbZ/4YUXcOvWLbz66qs4d+4cYmNjMW/ePEyZMkWuj0BEREQ1iOxLwUePHo3r169j9uzZyMjIQEhICDZt2iQmGaempkKpLI/BAgMDsXnzZkydOhWdOnVCQEAAXn31Vbz55ptyfQQiIiKqQRSCULeKsWRnZ8Pd3R1ZWVlwc3Ozynv8d8V+7E66ITmWMn+4Vd6LiIioLqjM97fs01K2qFcLL7m7QEREVGcxuLECViYmIiKSD4MbK/ho81m5u0BERFRnMbghIiIim8LghoiIiGwKgxsriGzvU3EjIiIisgoGN1bQ1k9/iVrOvSIZekJERFT3MLixgie6le487u7sIB5LTLsjU2+IiIjqFgY3VhDg4YzT7w7Bxlf7iMfS79yTsUdERER1h+zbL9gqZ0c7ONiV70ZepLXRJxEREVkPR26syN6u/PYeSL4lY0+IiIjqDgY31ST2WLrcXSAiIqoTGNxUk2I1t2QgIiKqDgxuiIiIyKYwuKkm3vVVFTciIiKi+8bgppo836+53F0gIiKqExjcWFlYsCcAoBFHboiIiKoFgxsry7pbuu3C8StZMveEiIiobmBwY2VnMnIAAF/vvChzT4iIiOoGBjdERERkUxjcEBERkU1hcENEREQ2hcENERER2RQGN1YW4OFcYZsStQA1t2cgIiKyCAY3VjZnRDvxcdK1HL3Xi0rUaP7WBjR7a0N1douIiMhmMbixsnb+buLjiMU70fW9rbhXVCIeO3k1W3x85c7dau0bERGRLWJwY2XODnaS5zfzCrHxRLr4/PjlO+Wv5RZUV7eIiIhsFoMbK7NX6t/ivILykZtZf54UHz/8+Z5q6RMREZEts5e7A7bOzk6hd+xa9j0UFqux+t9kGXpERERk2xjcWJmdQj+4+XR7Euqp7BGz8YzeayVqAXZK/XOIiIjIPJyWsjJjgYqhwAYA1h+9Ys3uEBER2bwaEdwsXboUQUFBcHJyQlhYGA4cOGDWeWvWrIFCocCoUaOs28H74GhfuVucnnXPSj0hIiKqG2QPbn7++WdERUVhzpw5OHLkCDp37ozIyEhcu3bN5HkpKSmYNm0a+vTpU009rR6pN/Pl7gIREVGtJntws3jxYkyaNAkTJ05Eu3btsGzZMri4uGDlypVGzykpKcFTTz2FuXPnolmzZtXY26rpHOhhdttT6dkVNyIiIiKjZA1uCgsLcfjwYURERIjHlEolIiIisHfvXqPnvfvuu/D29sazzz5b4XsUFBQgOztb8lPdRnb2N7utr5uTFXtCRERk+2QNbm7cuIGSkhL4+PhIjvv4+CAjI8PgObt378Y333yD5cuXm/UeMTExcHd3F38CAwPvu9+VNS68Kfq09DKr7bCOflbuDRERkW2TfVqqMnJycvD0009j+fLl8PIyL1iIjo5GVlaW+JOWlmblXuqzt1Pi+2fD0M7PrcK2Dna16ldCRERU48ha58bLywt2dnbIzMyUHM/MzISvr69e+wsXLiAlJQUjRowQj6nVagCAvb09zp49i+bNm0vOUalUUKlUVuh95f04KQwh72412aZE4O7gRERE90PWYQJHR0d07doVcXFx4jG1Wo24uDiEh4frtW/Tpg2OHz+OxMRE8efhhx/GgAEDkJiYKMuUU2V4uDhW2EatZnBDRER0P2SvUBwVFYXx48ejW7du6NGjB5YsWYK8vDxMnDgRADBu3DgEBAQgJiYGTk5O6NChg+R8Dw8PANA7XlupOXJDRER0X2QPbkaPHo3r169j9uzZyMjIQEhICDZt2iQmGaempkJpYPNJW1XCkRsiIqL7ohCEujVUkJ2dDXd3d2RlZcHNreIEX0sLmhFr8vUPH+uI0d2bGHztRm4B5qw/iTcj26BJQxdrdI+IiKhGqsz3d90ZEqkhBrXzMfl6idr4a93e34bYY+no+9EOC/eKiIjIdjC4qWYJqXdMvs7VUkRERPeHwU01u5FbYPJ1rpYiIiK6PwxuapiqJBSXqAUGRURERGUY3NQwX++8KHmeV1CM3w5fxp38QsnxorLknMJiNZq/tQHN3tpQbX0kIiKqyWRfCl7XzX24Pc5fy8H/9qUCADKy72HWuhOIbO8LDxcHfLjpDHadv6F33q28Qvi4OeGXQ+XbSQiCAIVCUek+XLqZhwmrDuKrp7uilU/9qn8YIiKiGoAjNzIb3zMI74/qKDn2/b5L+O83+/HQZ7sNBjYA8OgX/wIAfLR2Ef/10OUq9aHfR/FIvpGHwR/vNNrmVl4h6ljVACIiqqUY3FSzipaCm+vKnbsAIJmueuP3Yxa5tq5XfkpAl/e2IjiaU19ERFTzMbipZjOHt7Xo9X48kGrwePa9Ijz37UFcy7l33++x/ujV+74GERFRdWFwU82URnJiJvQMqtL1VPblv0J3Zwfxcad3tmDb6Wvo8UGcodOIiIhsFhOKq1nOvWKDx3VXQ5lDdyuHyX2bValPlaFWC1AqK5+0TEREVF04clPNjNWxUdnbVXNPSlU2SbiiIoRERERyY3BTzRztDd/ytNv5Vbpeo/oq8XFhseGNqU5cyTJ6/opdyRW+h797+YqsuX+fqkTviIiIqh+Dm2rWysfV4PHz13KrdL3WWnVpCo3sujn3r5NGz0++mVfhewQ0cBYfbzmZUYneERERVT8GN9XMWJG9sT2aVOl6u5PK6+B8GX/BYJtjl7NwLjMH94pK9F77cb/h1VbairWm0rTr6hAREdVEDG5kEBLoAQDY/9ZA8diUAS0kozCWVFCsxuCPd6LNrE1VOl87T+jy7buW6hYREZFVMLiRwbopvZAyf7hkFMTRXolPxoRUeG7fVo3u673TblU+t+fYZeM5O0RERDUNg5saxNGu4l9HkZGkYXNpKhub61xmjsnXr+Xcw02uoCIiohqEwU0NonKoeDn4gDamR26MLTXXUOu87uIofU/dQGXVnhSj17pXVIIeH8Sh6/vbxPe9cD0Xu85fN9kHIiIia2JwU4PYVbCj95gegXimV7DJNoXFapOjM2NX7MefiVcQNCMWR1Jvo4GLo+T1Yp3gp2lDF6PXuplXXngwr7C0OOHARf/g6W8OmFx+bsztvELsOn9dLwAjIiKqDAY3NYjKSA0cAPjvA00Q82gn2FcwdXW3qAR3C/VXRWl7dU0igNKdxXUDIbVgemQHKC/856BVqbi4RHreniTDu5mb8tBnu/H0NwewNuGK3mvHL2dh8neHcOF61ZbMExFR3cHgpgZpUM8RA9t4G3xtakQrs66x/cw1VDAAZJJaKM3L0Yy8GNoLq6gskLlXVJ7/k5El3aDzw01nKv3emkBr2q9H9V57eOlubDmViee+PVTp6xIRUd3C4KaG+WZCd/Hx1ql90SHADd2DGsCzXvn00ZQBzY2eLwgC/jQw8mFMG1/p8vP1iVfRa/52PPTZbqTdyjcYKJ1KzwYALNxyVjz21c4LOK61qsrSM0uaAaXkGxUXHSQiorqNG2fWQCnzh4uP10/pDYVCWvxvemQblKiBZf/oF+1zcrDDp9uTzH6vIp2qxl/sKD/3TEYOXFX6fyKjlu5ByvzhWH/0qnjswvVcXMu5p9fWkPSsu7h4PQ+9WniZ3U8iIiJzceSmhlMqFQarGs8Y2sZg+8OXblfq+heuS0dCmnuXbw8x6btDYg5On5amA5ETV7Jx8mq2+NzXRCXj8JjteGrFfgz7ZFel+kpERGQOBjc2ZvW/Kfd1fmLaHclzTaKwvVI/wJrQM0jy/N8L5UnED3Xyq/C9NNNbFdl2KtOsdkRERACDG5swPbK11a49Z33pppt2Sv0/Fe3l3tMjW6OF1qjPit0V7zZurue+YxIxERGZj8GNDdBONra0/LJl5dtOS0dP1GoBh7SmwDafzEBgA2lNHGtVLr5wPReCIGD2nycM5h0REVHdxuCmFnu+X3O093fDqJCAan/vAp1tII5dzkLMRuny70KtZOXse0UImhGLoBmxkja3tAoBmmv1nhScvJqN7/ZewvyNZyqs60NERHULg5tabMbQNoh9pQ+cDRTas7YidcV7XOUVlAcdS7aeN9jGriyXJ7egWHJc97m2vq0aIede+etHL9+psC9ERFR3MLixEU08jW+TYCkdA9zFx+Zs4KmdYLxyj5EcnLL6NS/877Dk8MGUW0av6+JoJ1miXp1ViwWBW0MQEdV0DG5sROqt/Ps639/d+NJtjeaN6omPC0sqDm4CPJxxr6jE5GonoSy62XVeul1DYANnAEBBsf6U093CEsk2EW//caLCDUO1FZeoEb32GL7fd8nscwDgqRX7EBy9Qa8aMxER1Sw1IrhZunQpgoKC4OTkhLCwMBw4cMBo2+XLl6NPnz5o0KABGjRogIiICJPtyTzrX+5dYZvn+jQTH2tPCxnj4miP8Jg4k6udjA2E/G9fKoJmxOJgsn7dnkVbz2HtkcuSY3sv3KywPxq7km7gpwNpmLXuRKXydfYklb7HAzFxZp9DRETVT/bg5ueff0ZUVBTmzJmDI0eOoHPnzoiMjMS1a9cMto+Pj8eYMWOwY8cO7N27F4GBgRg8eDCuXDF/ywFbZKioX3izhmaf7+liesXVxlf7wM3JQXyeb0ZQEHv8Km7nF5lscy4zx+BxTb2e/36zX++10+nZcHd2kBy7lW9+YvKt3PK2vx1OM/s8IiKqHWQPbhYvXoxJkyZh4sSJaNeuHZYtWwYXFxesXLnSYPsffvgBL774IkJCQtCmTRusWLECarUacXF1+7+mDeXALHu6q9nnKw0U6dPo1Ngdbf3c0KBeeUCRfdd00AKUjr5UpCp7UCkUwAPNpYHbLwfND1LyCstHnWKPp1e+A0REVKPJGtwUFhbi8OHDiIiIEI8plUpERERg7969Zl0jPz8fRUVF8PT0NPh6QUEBsrOzJT+2KM/ASIqbk/Gtw94b2d7k9eJe7yc+njGkdFRIO4n3h/2Vy1cxZszyfXrLwysiCNDLsdmddMNIa31ODuWry/ZdNJ64rO1aNvNsiIhqC1mDmxs3bqCkpAQ+Pj6S4z4+PsjIyDDrGm+++Sb8/f0lAZK2mJgYuLu7iz+BgYH33e+aqFvTBnrHDO1JpfFkd+P3ISzYE80buSK0iQf83Z3QI9hT73qbT8q7JYJmWwht2feKEH/2GoorSHa+V1T5ujhv/XG80ucQEZE8ZJ+Wuh/z58/HmjVr8Mcff8DJyfBqn+joaGRlZYk/aWm2mWNhZ2JayRCVvX5tHM3+UW8NawsAWPtCT+x+80HY29W8P5M9BkZqxn1zABNWHcTIpXtMnjv7z5OVfr9AnaX2untwVYdLN/MQNCMWDy6Kr/b3JiKqTWT91vLy8oKdnR0yM6WjAJmZmfD19TV57sKFCzF//nxs2bIFnTp1MtpOpVLBzc1N8mOLjqRWvBt458aldWrOfzDU4Ov73hqI9S/1QudADwClIzWmcnEM+fCxjnigmeEpQku6qVPZ2NnBTgw4Tl7NNpqoXFW6u6I/u/qgRa9vDk1y9UWdndyJiEhK1uDG0dERXbt2lSQDa5KDw8PDjZ63YMECvPfee9i0aRO6detWHV2t8XS/zFdP7C553iHADX++1Bsp84fDwchIjJerCp0ae5h8H80UlTEPdfLH/EeNB5tVNaZHE7w/qoP4fNf565LX7+pMNa09Yt7quS5NPMxqpzvFpxtcVQc/N2eLXetMRjZW7LqIIjPqFRER1TbGM06rSVRUFMaPH49u3bqhR48eWLJkCfLy8jBx4kQAwLhx4xAQEICYmBgAwIcffojZs2fjxx9/RFBQkJib4+rqCldXV6PvY+s6NfYQ82Ai2/ugf2tvAEDsK72x5kAapg/R3zm8e1ADHEypeMRH24Fk/QTclPnDJc+v51R9w8wuTTxwJPWO3vEPRnXAP+fKA5obuaaDC7WRAjq6lY8NvZchNWH/qgMmqjZX1pAluwAAh1JuV2pVHRFRbSB7MsXo0aOxcOFCzJ49GyEhIUhMTMSmTZvEJOPU1FSkp5cv1/3yyy9RWFiIxx9/HH5+fuLPwoUL5foINcIL/ZqLjz8b00V83N7fHe+N6iCpUaPRzs86U3QO9lX/s1r2X8NftEqlAn1bNTL7OneM1L15Ypl5q/B0vbO+8nk6utRqoVKVlE1JvXl/Fak1km9Ufoorr6AYW05mVCkxm4ioOsg+cgMAL730El566SWDr8XHx0uep6SkWL9DtZBSqcDFecOgUJheJaXtyp279/2+Qzvo50YFeEinTx5o5ol9F2/ht+fD8XgFwYW3m/FtIOyUCtgpFWYFCL8cuowFj3eusB1Qeh90+6zrms5olKHPXZHJ3x/GyatZ2PRaX70ihBXJvietK5SedRdNGlZtP7EcrWudrUJu0tgV+3E07Q7a+NbHptf6VqkP1pR2Kx9bT2ViTI8msmwqS0Tyk33khixHqVSYHdgAwMInSr/8+7c2f0SkR5A058bXjD2p1kwOR8r84egW5IlhHUuDgv8+0MRo+7jX+2GVTs6QRmVGPnaeu15xI1Q8ChL58U69Y/4VBEO6BEHAttOZSM+6hz90to4wx6mr0vpMnvVMV5TWNfLz3Ri0+B8AwJoD97di8GhZ4vaZDMsmbVvKmOX78O7fp/DhpjNyd4WIZMLgpg7zcHFEyvzhWD2xh9nn/Px/D0ieN6hg2wZdXzzVFSnzh5scuWjeyBUDWntj8ZPmjbwYM26ldM8xYzt6N6pv/DNcvJ5rcHTDWJAlCALWHrmMYZ/swuw/T4gJu5dvl4+SvfPXKfFx9NrjCJoRi3kbThv/IACm/XpU+j4G2twrKkHQjFgEzYiVfNbkG3k4ejkL56/l4kjqbXxQwXtpZGTdw/d7U5BfaHwfsbyC8teKS9TYfiYTq/Yk49LN8umu/MJinM3IMTpVaIogCLiRW7kcLs291mzhQUR1D4MbqhS9VUOV/OLRqG8gB0jXyJAAzHukIzabmPqIn9YfZ98fYvR17YJ+xgoPmhoMOnzJcMK1seBm/dGriPrlKE6lZ+O7vZfwybbzAIAEA3Vxjl/Owk8HSreo+HrnReOdQHkNIo3fD+uP/jz25b/i4+DoDeLjhVvOio+TMnMl5/Qzkcf0QEwcZv15Eu1mbzbaRnvvr5V7kvHM6kOY+9cp9PsoXjzebvZmRC7ZiZB3t+LY5TtGr2VIcPQGdHt/GzZwmwwiqgQGN3Rfvt1reBuGT/4TAgBo7VPf4OuPhAboHft4tHSkxk6pwNiwJmjta/gaa1/siSCvegYLEmq0eHsjUsqSZr/aecFgm43HM4wuiZ7+2zGDx0uMjAK9qzUqAwCf70hCQXEJXvkpQa/tiM936x375WAagmbEYs0B6b5cuvegb6tGuHw7H7HH0qEuC7RO6kxdaRJ+G2tNof2uMyVWrDZvKXhGluHtJ3K1doc3Z/n9mK/3mfV+gDQ36MUfjph9HhERgxuyioc7++OPF3vi9xd7Gnzdx80JXz/dFV6uKhyaGYGU+cPxSGjjSr2Hs4N5yaL9F8YDAM6kG84R+XjbObR8e6Nk+sXYFFagZ2mgsPfCTbHdlTt3IQgC0rPu4qUHW+idM/1X/QDpxJUsvWPnM3Pwxu+lbWeslW73oDvq9L99l9D7wx2Y8uMR/JFQGlSMCvGXtLmWXTqq9pXWqNB+naX8RQa2sTDkapbh5HMfrQRwc3JwDO2BZkjW3SKMWW5+IKShu4LL2O/R2m7lFVa4DQgRWQ+DG7ovg9v5GDyuUCgQ2qSBZLNNvXPb++LQzAh4uaqq9N66UzUVeaZ3kMnXNdMveQXFGLAwHtFr9feTupNfOpqgWULd/YNt6DV/O5779hDCY7Zjrs7IDVA6VaVr2T/6o0jLdxmfmtK9RxtPlO+9Nr8scdaznrRN3492GL2ehqG6RSeuZOkVhTSWL7M76QaCZsQarJBtKlenIp9vP48TVyq/yW1BkTSg+CLe8GidNZ24koUu721Fi7c3Vvt7E1EpBjd0X754qkvFjaxEO/1n/1sD8cl/QnD+g6H4/QXDo0Xm/kf8+qNXkXIzX8yH0fbqwJbi4+OXs8RignFnrpnfcQB/H9PPIdl2WnqNwmK1mCAcZqIy9PWcAqTdysfhS1Ur8ncjtwBqtYB7RSU4l5mDhz7bjcE6K8Ry7hUj5Uae0RyrR7/4V+9Yu9mbq1wB+bSRUbaKFBRLR24+2nzWSEvr2V7JvwUisrwaUeeGapekD4biRm6hWcvALe2dEe3E1UYtvMvzUHzcnDAypDSPp6uBHdK3nMzAca2poAvzhqH5Wxv02r371ymcv2b4i/Wvl3rjzt3yEQxDOTP345bOlg4xG8tXNcVWkFDbZ0HFozTGnM3IQczG0yZHSl5dkwgA+OCRDkbbGKK7LYa5dhvYGNUct/OLKm5kZdoB1p38QnhUckWhtviz1/De36fw0ROd0aWJ/t91TXAnvxBODnZwMnOamKg6cOSGKs3eTilLYAMAE3oF48isQbg4b1ilzpv8/WFk3S394hvczgd2SgXCmzXUa7dyTzJ2ndf/Yv375d7o2NgdjtW4Q/qqPSn3fY3MbMOJwNqeWrHf7Ckgc2sHaehOE7XwrniLFGNTYOYsJZ+x1nACeHWqzDYhugRBkIx2TVh1EBeu5xkcGatIiVrAvA2nzfobqKqMrHsIeXcr2s8xvqKOSA4MbqjW8aznWOndygHg2OXSkRtvt9LclL0Xb5p9btuyrSruZ2sJS3isS+WSrsPmxVXcqBLa+FZuyw7dBF+1GUUYM7MNT31p1woyJsHMvcJMuXw7H38fu2pWX3UJgiAJFD/ffr5S5wdHb0DLtzdK6gdV1Rc7kvD1zotG/wYskWy99VRp7leJWsDwT3ch1wL9JrIEBjdkk868Z7z2zfHL+iuVKmJXFkw5KM37v0w3A1NjltDa1zKbw9Z3qtqM9Cdxlfuy1s15KS4LGAqKSzD150S9Je8AjH5Bvv2HfoK3NkOBRERbwwnvpvT+cAde+jEB7/6tnxxekX0XpXlPiQbqGxmjvR3KE8v26uUPmaNELeBaTulIzaKt58Tj13RGb/IKihEcvQFBM2LFY7P/PIGgGbG4cF1aC8kU7WnHk1ezK7Vk//Pt5xE0I9ZgBfDqEvVzokX2jaOah8EN2SQnBztcmDcMXq6l+Q7dmjZASKAHAOCF/vrLtc3lYF/xiNGDbbzx2dhQJMdIp86aNaoH7/pVWxmm0bqSIyfGTI1oZZHrVER3pVjqrXxsOZmB1jM34Y+EK5ix9rjeCIKxDTlPXDU+dVZcosbCLef0jpeYWcdHQ3vqqyoVjjWBhUbKzdJ9rh5cFC9uW2HMr4fKt8U4lZ6NdQkV1w0CSrfmGLgoHrvOX8fIpbvR44M4vWKJPebF4eqduyhRC7hbWCKZRtJMW31XVrNq4KJ/sElrNZ4px3WmMyszban5fZ3NzLHYhrKVkZl9D2sTrmD1vyl6+W62Rq0W8OgXe9B6Zt1ZwcfghmyWnVKBN4e0AQC4qOzF/4rWLCHf9FqfSl/zVgU5FMkxw7ByQnf4uTvrVXN+vm9zceRCY+vUvlgzWbqlhSnmjLicmBtp8vXpka3xdHhTs9+zMn5/IRyfjgk12Wby94clz3VHd64YmX7yNbGpqiafSpexCtO6/ky8ghNXsnA9R39K7F5RCdRqAVn5RbirVafH0LSV7ma07s4OmPTdIVy8nocpP5aOagiCIK6C015yv2JXsuRc3efGDPt0Fy5cz8PT3xwQp8QWbjmHljr5TT3nb8fIpbuxLlEaNKXe0t9b7fn/HdY7ZshfBsocVEX82epfYSbZQPY+90nLzL6HoBmx2HSiZlbSvpp1F0dS76CgWI3bNh7IaTC4IZumCTC0/4tSXTZS0EprtdXnY01/IWv0MLEkW2Wv1AtoDr4dgQfbeOOLp7rgiW6N8WS3QMnrLX3qw08nOXvuw+2Nvkc9R2lw085POpLj4eIAV5U9erXQT5aeMqA53h3ZHlMGtICDnRIfPd7J6PtUVWhgA6O1j4zRrUVz+bbhjUw1m66m3szHB7GnJFWTP94mHbVp1qgeACD7nvEckAvXc/HVPxewdEcSXl2TiIc+241BOlMkB5Jvoc2sTWj21gZ0fncL2s7ehBK1gIGL4tHhnc16Ac4XO6SfRTvo0uQMaX/e/64o375Cdzru/DXp9JBu3aBRS/dIppUk75tfqHc+AJy4kq03qvPEsr0Gr1ERY0UKz2SYTk6/nlOAj7dKf1/m5FNZmnZ+lqFaVBXRHnHU5DU9/7/7q6RdXKI2OnJ5P7Lvlv/tFNaR4pIMbsimfWogR0STJ6BUKnBh3jCc/2AoHurkjwWPlX/Zv2yg0jBQulLs3ZH6wceJuZEG83wa1Vdh5YTuGNbRDwqFAj5u5dNSvVt4AYDeUuFRoQEY00MaBGm4OJYvt/VxU2HDq9LRJ02RQUOboU6PbINx4UHi8ye6GX4Pc0S09caOaf3x0oAWkvuhVCoMLgk2VcxRl6EaQACwfFcyBEHAmOX7sHxXMv5Pa3Thf/ukuTuhgeU5T3cNVEXOLyzGwEX/IGbjGZO1cJ78Sv+L//cjl3Hheh7yC0uwaOtZrDmQitt5hSgqUZuVUKv9RX4tp8DsjUEnrDwoPr6dV2gyn+eSgdEYjbwCy3x55hgJHPdfNF1vqfsH2/Ryt+bIkPeivbWKoZpWpgTNiEVw9AZ8W8XNWdVqAecyc8Tg+N2/TqHz3C1o8fZGtJm1SRK0Fhar73vFm/Z0qRxTgHJgcEM27YFm+iMt2l92dkoFHMqWdz/U2U88/lpEKzEZ9dtnpIHCuPAgHHtnsOSYq8peb9TGEO1/WDS1XOo5SoMBd2cHxDzaCUdnD0bCrEGS1+qp7LHsv10QFuyJdVN6GX0fh/tcsj77oXYASpfAG7L0qS4I9qqHaZGtEdne1+S1XuzfHCfmRuKrp7sabRN7LB2LtpzFnfxCcWRNo43WvlrB0RvEqZ+jaXdwJiMb53WqKf/4XBimRZbnFLWdvUl8LAgCjqTexvrEqk+nvKH1pbh0xwXMWHscoe9tRUszKxLrfpF2e38bACBAaw8wQw6k3MKcsqTf0Pe2mmzbMcDd6Gu6oxRPP9AUJWqh0mUOjNUwWv1vSpVWmllK1C+J6PreVpNf4vezUkz7s81Zf1JvBMtU4crUm/koUQuYuPogBn+8E2/9cRwnrmRh5Z5kvanVG7kFGLt8H1rN3IiweXHiQohrOffwy8E0g0G7MX8dLf8PhmX/XKhUsrpaXT6NKtd2JlXBIn5k06IGtcYvh6SbRRqrteLiaI+L84aJy8xXjO9m9LpuTg7466XeGL/qAH6aZH7OjPZ2B5ovbXsjXyruLvo7p7s42mFIBz8M6VAeiB2dPRid392i1zZqUCssLhv+NxTkAaVF+d7+4wTWvthTUkvlmd7BeKZ3MADg7PtDUFisRsd3yt9De7NSHzcnbH6tL9ycDf9zotlkVDcI0i7IqMlHOZ2egyCveki5mY+xYU3Qr1Uj3CsqEYsI6hqyZBfqa40KNXBxQM8WXnpLqb/ZnYz3qrD6ydK08zy0Ldh0Ri9fxxBjG9XqMlSryZjv913C9/uk19WO09VqAUVqtd4GtYbyk4DSrUmavbUBKfOH671m7ihVVV25c1fcwHXNwVQ8FWY4t6yguOpTM7rTOrq/k4JitcH/uPj23xS9Eao1B9OMbj+z5kAq/r1QXq5ixOe7kTJ/OHp8UDoF9sbvx7DrjQEI9HSpsM/aG+Z+t/cS7hWVYMHjnU2cUW7F7vJtYeZvPIPoYW3NOk9uHLkhm+ZhIEAwVMFYozL1czo2dseRWYOM7lpuyJgeTcTH5mw0CQA/TgoTH6sM1Nlxd3FAe//S3JtFT5T/g6U9tTZnhOE8nqfCmuLivGGS6rdNG0r/sVTZ26G+k/591Nbatz783A2PPHRu7GHweBcDv4dtpzMRf7Y0PyrAwxmR7X0rTPbM0QpkGpWtRtOdGqvuwOa/DzQxeFw7QNQmxx5YpghC+QhFs7c2oPXMTXojTob2E6tIVYoRAqUBk+5oUNi8bQiaESuZnus1f7v4+O0/TkAQBKTdytcbcTA0zXM7rxDzNpzWGwnUdVJn1Z7u35busnugdKTI2NTb5zuSDB6/Y0a17apWJtf9Dz5TvtbaeFd3z7majMEN2TTdzTVberuaNX1kLfW0RhmGdywffVlQltyrHcho9GzuhaVju2D1xO5G+x77Sh8kxwzDY13Li/wpFAocnT0Ym1/rKxYhNEQT0O2cPgCT+zbDhlcMryIzNkVVkWFan1N7Oq9TYw842Bn/Xawo20g0oIHp6Rpt/mVTO3ZVKPKo7fl+ze/r/E5GArqqaF6WHF3d4s9dw79a22DobiRbUe7G1Tt3EbPhNE6nlwcDHQLMK2UwcFE8gmbEorikdH+1AQvj0eytDShRC+KUiqbY46ilewAAfyTof2EHR29AnwU7EBy9AUlaCda6id8AEPreVny98yL+83X5bvTLd14Up2SCZsTiXlGJwc1mtQ35ZJfk+eCP/0FwtP5WLxVZsVt/tZyhBPKqTgEWmjl6NUhrgcCOs5WrUC4nBjdk03S/5IxNAVUX7akb7amiJ7sFImX+cPRs7mXwvOGd/NC/tbfJaxsKfNxdHMweWWrS0AVvDWsrCcC0dQhwx4m5kWZtfaGbk6Th5uSAlPnDxSkL7QBPl2ZoX3eFmSmzynKF7tf9Fku05D5LW6f2s9i1tL03yvQ+Yc+sPoSxWqu5AEhWWhWVlH6pOtop0aell95Kwme/PYSvdl7EUK0v++aNjN/XoBmxmLfhNARBwIXreQCgt7N687JRJN2ckR1nr2Hqz0dNfp6Ixf8gu2xa8GetmkK6bpYtlb6ZW4APNpyWvNZm1iZ8uOmMyffRDhoEQcC5TPOLIlbF9bKpvrRb+fjFwOfSrb2kMfn7Qyav+/JPCQiaEYufDhi/VzUZgxuyabpf+Nr/FSkH7bn4J7tXfbWSXFxV9mZN3ekGMcasM5HYO7FXac5PZZKjTX15GtLe3w0p84fjwrxh+E/3QDRt6ILNr/WFs0PF6YgJswbhwrxhOPPeELT2kQaQ5iRe/jz5AYNJxAPblAaxrX3qY/vr/Uze73PvD8W6Kb3w+wvhUCog1nWqyO8v9MTTD+jnozwSGmDyvF+1pjM0ibSPdQ3A98+G6ZUA0P7/2is/JSD2WDo+2y6dgvlTJyn+650XseF4xQUEW8/cJHk+cdVBIy2lOhmZFjTklJF/KzSFQU3JuVeEb/9NQdot6y9x1yQW91mwA2/8dgwhOvl3Z9INTyXFmxiFKS5RW6yGkVwY3BBVoyaeLmhUX4Vgr3rVuglnTTXIRE2cir5oK2JO7SLNVJudUoH5j3XCP9MHoLVvfb3pTIWivHaORoN6jrArW/q+eWpf7HpjgPiam7ODuNQfAE6/q18mIKxZQ+yZ8aDe8W8mdEfK/OHYPLUvmpUFa9plCrQ52isREuiBrk09cXLuELzQXzqd1ruFF/bMeBDvjGiHC1ojbk3KklC1pwWbN6qHj0eHwNSs7ff7LmHG78dwK68QX/5TOrWjWeVjaquL9UeviknjGuc/GIrOZVXDtem2szTdqR3t8gwaOfeK8PQ3BwyeH9Cg4gTeju9swZz1J9H3o6rlxFRGsVqN5Vp5Mbq5OsUmqnRPWHUARSVqTP/1KH4/XB64mloar12HJ7egWPb/YDSG/7oSVSMHOyX2vPkgtkX1kzX3p6b48qkuRl/T/uJd8Hgn9GzeEMkxw7B1al8cnT0YSR8MxS//F270/Ic6+euNHKXMH44z7w1Bv1aNsOy/XYz+DtycpQnUFz4YVuH0UGOt3KDOjT3w9biu+HxsKI6/MxjOjsanqeY/2lF8bKy685PdA8WRsPhp/dGpsTu2RUn7o3mP+Gn9y6/9WEcEeDhjQq9g2CkV2DGtP/5+ubeYeK2ZWgJKV70BqHDacc3BNHR5byvyy0YMNCMtQV71zBrVAIDwZg3vu1yBJTz9QFODuSdDluwy0LqUsW00dEfvqsvT3xzQmz7TTpjWfD5fNydM7BUkaRd/9jrmrD+JXw9fxuu/lk/rXbljvK7OjjPl1aSj1x7H0E92YeupzPv5CFYh/18XkZX99VJ5Imy/Vo1k7EkpR3vlfSe82gp7O6XR6St7rU1Kn+wWiB8nPQCFQoGWPvXh7uIAezslegR7Yu2LPQGUbithDicHO3z7TA/JcnpduuUClEoF7JQKzBxeugz26Bz9nCKFQoHE2YPw74wH4VnPES6O9niok7+40uyPsn4CpX8DGv/p0QTn3h+KbVH98HBn/wr7H+RVD+tf6m20pIF2YKYbpAV71UMHrRo42jWWvigLNHUDvqYNXfDxaOPLhp/QSmI/NHOQ0YR0bQusUB3bFN193jQa1HNE16b6ZRLMWZbfsF55IDdzeFsMaGM6J06bocTqU+9GInpoG6wYV16CYlx4U3z0eCcM7yT9W/1Gq0xFepZ+IDJ+5QF8Gncet/MKkXKztKCjt5sKc0a0l1wfAH7cXz5KEzQjFn8dvYoEEyvhYo+n42LZ5qqaqatJ35nO35ED69yQzWvnX/4PyYyh5uUkkPzsTKyk0talSQOTuT1/v9wbD322G68MbGn2e3tqfXFNGVA+1fNcn2Z4rk8zo+d5uDjCw8isRWiTBjjz3hAcvnQbPZtLt8dwtFcaDVYqy7OeI3oEeUKhKM19MuXX53ti2Ke7xL5rrJn8gLhqqHcLLzjZGx95erRLY8lz7f+/GdOokhvIDuvoiwvX8nBWZynyV093xf/p7FWma3in0urgDVwccFtnykYQBPRv3QjbThsfeVgyOgShTTzQ76N4yfHYV/rAV2vrlOISNZb9Y3hJ/45p/fHIF3sw75GO6BHsCQ9nB5zJyMFDn+0W27g42uP/+jVHmlZ16a2nMvHuyA6SukW+bk4YWMFu92cycnAmIwfLd10Uy0ccKysCqHIwPabx8k8JBo93auyOY5ez8PexdPx9LB1DO0jrVmmm+7xcHXFo5iBDl6hWHLkhm2enVOCxLo3Rq0VD2YaOqfKcLbTiqEOAO1LmD0fUoKrthN7B33i138pycrBDrxZeVp+S/OX5cPxsYspOo52/G869P1RvZOOBZg3xTK9gtPNzwwePdDS5nYO9gSB0/UvGq2cD0tVk0wYb/r1oX+PzMV2weWpfnHt/KP6vX3lwqV3AsVNjd5ycG4nDMyMk11k6tnRE6sisQXq5VD2be+GJbo2x8InO2Bc90GA/Rob4o2lD/eX4vjp7whlbiVnfyR7BXvWQOHswhnX0g5erCvZ2SskImvYWJtpF+TQF87oHldeEWv9y6X05bmRForace8UoKCqdltLkFoUF6+87V5EeQZ5icKSx0cjO8TdyC40WeKxOHLmhOmHRk+ZV4yT5jAzxx59aq6fknrp7vGtjHE27U6nphtrI0UBhSACYPaJ8Wf3YsCaYv9HwEmhDv6dOjT0ko2mCIBit9fLSgy3x0oMtMeWHI4g9XrpNwJgegejU2AOf/CcEPm5O4ooxR3slpg1ujdBAD3QL8pRsQZB1twj1VPaop7LH8XcGI2xeHN4bWb7cXaFQ4MTcSBSrBeQXFCPpWi7Cy0bQHi+bWjM0ElTVQPTs+0Owek8K+rU2PhWuGQ0ZojMKkjBrEM5m5uCBZqX9GxUagFl/noSHiwO865cGVbqFNZs2dMFXT3fVyxfyqOeAnIJiTBlQWtTT2O/bkKkRreDqZI9HQgPQpYItP7R1/2AbHO2VOPf+ULPPsTSO3BBRjTBzuGVq1FjKwic6Y8vUvhatWVNbmZresjPjy1+hUIiryVr5GJ5+054ynvdIaZL1yJAA8Qtew8FOiSEdSkdAtCuQ38otFB/Xd3LAqXeHSIpaAqUjRq4qe3i7OaFnC/2aUpHtfbFz+gAMKdsqRHsD261T+1b4OU+9GwmFonS0SGVvh//r1xxtfI1P0617sRfOfzBUDFg0GtRzlHzu+mWlFRJnS0drtPe9++vl3nB31v89aZaja+ewHZ0zGG8Pa2sylwoABrb1xrO9gyXTtOYqLFYbrbFTHThyQ0Q1gnYehm4CpVy4oq1cyvzhEAQB7edsFldKAeaPsAV6upjMjQr0dMGBtweivsrB7Pvu4lj+FZZjxo7s5mjS0AXLDGzy2tKnPnoEe+JA8i2xFpGh/iTHmK7tpE2pVECJqv+N9WvVSHJPTQWh2lNy7s4OmNS3dHrvkdDyAHDG78ew5mB50b5ArWXvx94ZbLBO0NE5g9F5ruH6QbpBW3VicENENYadUoEStYD/GtnwkOSlUCjw5X+7YvzK8howuhtJ3o/KfhlqB1bmFjC8Hz9PfgCXb9+VLPuvaTTBzuw/T+A7rU09b+cXGjtFNP+xTpge2Rpdy3aq1968V1OY815RCdrMKi2i+OaQNnB3dsCL/ZvjdHo2VozvjpnrTiCooQsm6Cw7r24KoTbtYW4B2dnZcHd3R1ZWFtzczNvnhIiqx7Xse0i+kYewZpVPeqTqoZs/c2hmhNGdravDvaISpN7KRysuFpCYue44/revfJm3s4MdTr+nX0yyKm7kFqC4RNBLqra2ynx/M+eGiGoMbzcnBjY1nPaU0aNdAmQNbIDSPBoGNvom95FWq9atsH0/vFxV1R7YVFaNCG6WLl2KoKAgODk5ISwsDAcOGC57rfHrr7+iTZs2cHJyQseOHbFhQ+V3XCUioqrZFz0Q8x/tKCb+Us3TpKG04NKy/+rnEdky2YObn3/+GVFRUZgzZw6OHDmCzp07IzIyEteuXTPY/t9//8WYMWPw7LPPIiEhAaNGjcKoUaNw4sSJau45EVHd5OvuhP/0aMKVZDXcO2XL+V8a0EJSP6cukD3nJiwsDN27d8fnn38OAFCr1QgMDMTLL7+MGTNm6LUfPXo08vLy8Pfff4vHHnjgAYSEhGDZsmV67QsKClBQUF5QKDs7G4GBgcy5ISIiqkVqTc5NYWEhDh8+jIiI8oqSSqUSERER2Lt3r8Fz9u7dK2kPAJGRkUbbx8TEwN3dXfwJDAw02I6IiIhsg6zBzY0bN1BSUgIfH+k+GT4+PsjIMFzaOSMjo1Lto6OjkZWVJf6kpaUZbEdERES2webr3KhUKqhU8mbzExERUfWRdeTGy8sLdnZ2yMyU7siamZkJX19fg+f4+vpWqj0RERHVLbIGN46OjujatSvi4uLEY2q1GnFxcQgPN7yjbXh4uKQ9AGzdutVoeyIiIqpbZJ+WioqKwvjx49GtWzf06NEDS5YsQV5eHiZOnAgAGDduHAICAhATEwMAePXVV9GvXz8sWrQIw4cPx5o1a3Do0CF8/fXXcn4MIiIiqiFkD25Gjx6N69evY/bs2cjIyEBISAg2bdokJg2npqZCqbWbac+ePfHjjz9i5syZeOutt9CyZUusW7cOHTp0MPYWREREVIfIXuemunFvKSIiotqn1tS5ISIiIrI0BjdERERkUxjcEBERkU1hcENEREQ2hcENERER2RQGN0RERGRTZK9zU900K9+zs7Nl7gkRERGZS/O9bU4FmzoX3OTk5AAAAgMDZe4JERERVVZOTg7c3d1NtqlzRfzUajWuXr2K+vXrQ6FQWPTa2dnZCAwMRFpaGgsEWgHvr/XxHlsX76/18R5bl5z3VxAE5OTkwN/fX7JzgSF1buRGqVSicePGVn0PNzc3/p/Kinh/rY/32Lp4f62P99i65Lq/FY3YaDChmIiIiGwKgxsiIiKyKQxuLEilUmHOnDlQqVRyd8Um8f5aH++xdfH+Wh/vsXXVlvtb5xKKiYiIyLZx5IaIiIhsCoMbIiIisikMboiIiMimMLghIiIim8LgxkKWLl2KoKAgODk5ISwsDAcOHJC7SzXCO++8A4VCIflp06aN+Pq9e/cwZcoUNGzYEK6urnjssceQmZkpuUZqaiqGDx8OFxcXeHt7Y/r06SguLpa0iY+PR5cuXaBSqdCiRQusXr1ary+28DvauXMnRowYAX9/fygUCqxbt07yuiAImD17Nvz8/ODs7IyIiAicP39e0ubWrVt46qmn4ObmBg8PDzz77LPIzc2VtDl27Bj69OkDJycnBAYGYsGCBXp9+fXXX9GmTRs4OTmhY8eO2LBhQ6X7UhNVdI8nTJig9zc9ZMgQSRveY+NiYmLQvXt31K9fH97e3hg1ahTOnj0raVOT/l0wpy81iTn3t3///np/w88//7ykTa2/vwLdtzVr1giOjo7CypUrhZMnTwqTJk0SPDw8hMzMTLm7Jrs5c+YI7du3F9LT08Wf69evi68///zzQmBgoBAXFyccOnRIeOCBB4SePXuKrxcXFwsdOnQQIiIihISEBGHDhg2Cl5eXEB0dLba5ePGi4OLiIkRFRQmnTp0SPvvsM8HOzk7YtGmT2MZWfkcbNmwQ3n77bWHt2rUCAOGPP/6QvD5//nzB3d1dWLdunXD06FHh4YcfFoKDg4W7d++KbYYMGSJ07txZ2Ldvn7Br1y6hRYsWwpgxY8TXs7KyBB8fH+Gpp54STpw4Ifz000+Cs7Oz8NVXX4lt9uzZI9jZ2QkLFiwQTp06JcycOVNwcHAQjh8/Xqm+1EQV3ePx48cLQ4YMkfxN37p1S9KG99i4yMhIYdWqVcKJEyeExMREYdiwYUKTJk2E3NxcsU1N+nehor7UNObc3379+gmTJk2S/A1nZWWJr9vC/WVwYwE9evQQpkyZIj4vKSkR/P39hZiYGBl7VTPMmTNH6Ny5s8HX7ty5Izg4OAi//vqreOz06dMCAGHv3r2CIJR+0SiVSiEjI0Ns8+WXXwpubm5CQUGBIAiC8MYbbwjt27eXXHv06NFCZGSk+NwWf0e6X7xqtVrw9fUVPvroI/HYnTt3BJVKJfz000+CIAjCqVOnBADCwYMHxTYbN24UFAqFcOXKFUEQBOGLL74QGjRoIN5fQRCEN998U2jdurX4/MknnxSGDx8u6U9YWJjwf//3f2b3pTYwFtyMHDnS6Dm8x5Vz7do1AYDwzz//CIJQs/5dMKcvNZ3u/RWE0uDm1VdfNXqOLdxfTkvdp8LCQhw+fBgRERHiMaVSiYiICOzdu1fGntUc58+fh7+/P5o1a4annnoKqampAIDDhw+jqKhIcu/atGmDJk2aiPdu79696NixI3x8fMQ2kZGRyM7OxsmTJ8U22tfQtNFco678jpKTk5GRkSH5nO7u7ggLC5PcTw8PD3Tr1k1sExERAaVSif3794tt+vbtC0dHR7FNZGQkzp49i9u3b4ttTN1zc/pSm8XHx8Pb2xutW7fGCy+8gJs3b4qv8R5XTlZWFgDA09MTQM36d8GcvtR0uvdX44cffoCXlxc6dOiA6Oho5Ofni6/Zwv2tcxtnWtqNGzdQUlIi+SMAAB8fH5w5c0amXtUcYWFhWL16NVq3bo309HTMnTsXffr0wYkTJ5CRkQFHR0d4eHhIzvHx8UFGRgYAICMjw+C91bxmqk12djbu3r2L27dv14nfkeZ+GPqc2vfK29tb8rq9vT08PT0lbYKDg/WuoXmtQYMGRu+59jUq6kttNWTIEDz66KMIDg7GhQsX8NZbb2Ho0KHYu3cv7OzseI8rQa1W47XXXkOvXr3QoUMHAKhR/y6Y05eazND9BYCxY8eiadOm8Pf3x7Fjx/Dmm2/i7NmzWLt2LQDbuL8Mbsiqhg4dKj7u1KkTwsLC0LRpU/zyyy9wdnaWsWdEVfOf//xHfNyxY0d06tQJzZs3R3x8PAYOHChjz2qfKVOm4MSJE9i9e7fcXbFJxu7v5MmTxccdO3aEn58fBg4ciAsXLqB58+bV3U2r4LTUffLy8oKdnZ1edndmZiZ8fX1l6lXN5eHhgVatWiEpKQm+vr4oLCzEnTt3JG20752vr6/Be6t5zVQbNzc3ODs715nfkeazmPqcvr6+uHbtmuT14uJi3Lp1yyL3XPv1ivpiK5o1awYvLy8kJSUB4D0210svvYS///4bO3bsQOPGjcXjNenfBXP6UlMZu7+GhIWFAYDkb7i2318GN/fJ0dERXbt2RVxcnHhMrVYjLi4O4eHhMvasZsrNzcWFCxfg5+eHrl27wsHBQXLvzp49i9TUVPHehYeH4/jx45Ivi61bt8LNzQ3t2rUT22hfQ9NGc4268jsKDg6Gr6+v5HNmZ2dj//79kvt5584dHD58WGyzfft2qNVq8R+48PBw7Ny5E0VFRWKbrVu3onXr1mjQoIHYxtQ9N6cvtuLy5cu4efMm/Pz8APAeV0QQBLz00kv4448/sH37dr3puZr074I5falpKrq/hiQmJgKA5G+41t/f+0pHJkEQSpe7qVQqYfXq1cKpU6eEyZMnCx4eHpJM87rq9ddfF+Lj44Xk5GRhz549QkREhODl5SVcu3ZNEITSZYBNmjQRtm/fLhw6dEgIDw8XwsPDxfM1SxIHDx4sJCYmCps2bRIaNWpkcEni9OnThdOnTwtLly41uCTRFn5HOTk5QkJCgpCQkCAAEBYvXiwkJCQIly5dEgShdGmwh4eH8OeffwrHjh0TRo4caXApeGhoqLB//35h9+7dQsuWLSXLlO/cuSP4+PgITz/9tHDixAlhzZo1gouLi94yZXt7e2HhwoXC6dOnhTlz5hhcplxRX2oiU/c4JydHmDZtmrB3714hOTlZ2LZtm9ClSxehZcuWwr1798Rr8B4b98ILLwju7u5CfHy8ZClyfn6+2KYm/btQUV9qmorub1JSkvDuu+8Khw4dEpKTk4U///xTaNasmdC3b1/xGrZwfxncWMhnn30mNGnSRHB0dBR69Ogh7Nu3T+4u1QijR48W/Pz8BEdHRyEgIEAYPXq0kJSUJL5+9+5d4cUXXxQaNGgguLi4CI888oiQnp4uuUZKSoowdOhQwdnZWfDy8hJef/11oaioSNJmx44dQkhIiODo6Cg0a9ZMWLVqlV5fbOF3tGPHDgGA3s/48eMFQShdHjxr1izBx8dHUKlUwsCBA4WzZ89KrnHz5k1hzJgxgqurq+Dm5iZMnDhRyMnJkbQ5evSo0Lt3b0GlUgkBAQHC/Pnz9fryyy+/CK1atRIcHR2F9u3bC7GxsZLXzelLTWTqHufn5wuDBw8WGjVqJDg4OAhNmzYVJk2apBck8x4bZ+jeApD8f7Ym/btgTl9qkorub2pqqtC3b1/B09NTUKlUQosWLYTp06dL6twIQu2/v4qym0FERERkE5hzQ0RERDaFwQ0RERHZFAY3REREZFMY3BAREZFNYXBDRERENoXBDREREdkUBjdERERkUxjcEBERkU1hcENEVRYUFIQlS5aY3T4+Ph4KhUJvozxLW716NTw8PKz6HlUxYcIEjBo1Su5uENk8VigmqkP69++PkJCQSgUkply/fh316tWDi4uLWe0LCwtx69Yt+Pj4QKFQWKQPhty9exc5OTnw9vYGALzzzjtYt26duEGgtaWkpCA4OBgJCQkICQkRj2dlZUEQhBoZeBHZEnu5O0BENYsgCCgpKYG9fcX/PDRq1KhS13Z0dISvr29Vu2Y2Z2dnODs7W/y6hYWFcHR0rPL57u7uFuwNERnDaSmiOmLChAn4559/8Mknn0ChUEChUCAlJUWcKtq4cSO6du0KlUqF3bt348KFCxg5ciR8fHzg6uqK7t27Y9u2bZJr6k5LKRQKrFixAo888ghcXFzQsmVLrF+/Xnxdd1pKM320efNmtG3bFq6urhgyZAjS09PFc4qLi/HKK6/Aw8MDDRs2xJtvvonx48ebnN7RnpZavXo15s6di6NHj4qfe/Xq1QCAO3fu4LnnnkOjRo3g5uaGBx98EEePHhWv88477yAkJAQrVqxAcHAwnJycAACbNm1C7969xT499NBDuHDhgnhecHAwACA0NBQKhQL9+/cXfwfa/S4oKMArr7wCb29vODk5oXfv3jh48KDe/YqLi0O3bt3g4uKCnj174uzZs0Y/OxExuCGqMz755BOEh4dj0qRJSE9PR3p6OgIDA8XXZ8yYgfnz5+P06dPo1KkTcnNzMWzYMMTFxSEhIQFDhgzBiBEjkJqaavJ95s6diyeffBLHjh3DsGHD8NRTT+HWrVtG2+fn52PhwoX4/vvvsXPnTqSmpmLatGni6x9++CF++OEHrFq1Cnv27EF2djbWrVtn9ucePXo0Xn/9dbRv31783KNHjwYAPPHEE7h27Ro2btyIw4cPo0uXLhg4cKCkv0lJSfj999+xdu1acVorLy8PUVFROHToEOLi4qBUKvHII49ArVYDAA4cOAAA2LZtG9LT07F27VqDfXvjjTfw+++/49tvv8WRI0fQokULREZG6t2vt99+G4sWLcKhQ4dgb2+PZ555xuzPT1Qn3fe+4kRUa/Tr10949dVXJcd27NghABDWrVtX4fnt27cXPvvsM/F506ZNhY8//lh8DkCYOXOm+Dw3N1cAIGzcuFHyXrdv3xYEQRBWrVolABCSkpLEc5YuXSr4+PiIz318fISPPvpIfF5cXCw0adJEGDlypNF+rlq1SnB3dxefz5kzR+jcubOkza5duwQ3Nzfh3r17kuPNmzcXvvrqK/E8BwcH4dq1a0bfSxAE4fr16wIA4fjx44IgCEJycrIAQEhISJC0Gz9+vNjv3NxcwcHBQfjhhx/E1wsLCwV/f39hwYIFgiCU369t27aJbWJjYwUAwt27d032iagu48gNEQEAunXrJnmem5uLadOmoW3btvDw8ICrqytOnz5d4chNp06dxMf16tWDm5sbrl27ZrS9i4sLmjdvLj738/MT22dlZSEzMxM9evQQX7ezs0PXrl0r9dkMOXr0KHJzc9GwYUO4urqKP8nJyZIppqZNm+rlFp0/fx5jxoxBs2bN4ObmhqCgIACo8N5ou3DhAoqKitCrVy/xmIODA3r06IHTp09L2mrfUz8/PwAweU+J6jomFBMRgNJARNu0adOwdetWLFy4EC1atICzszMef/xxFBYWmryOg4OD5LlCoRCna8xtL1TDIs7c3Fz4+fkhPj5e7zXt1Uy69wUARowYgaZNm2L58uXw9/eHWq1Ghw4dKrw3VaV9jzSrzEzdU6K6jsENUR3i6OiIkpISs9ru2bMHEyZMwCOPPAKgNBhISUmxYu/0ubu7w8fHBwcPHkTfvn0BACUlJThy5IhkiXVFDH3uLl26ICMjA/b29uLIizlu3ryJs2fPYvny5ejTpw8AYPfu3Xrvp+mrMc2bN4ejoyP27NmDpk2bAgCKiopw8OBBvPbaa2b3h4j0MbghqkOCgoKwf/9+pKSkwNXVFZ6enkbbtmzZEmvXrsWIESOgUCgwa9YsWUYLXn75ZcTExKBFixZo06YNPvvsM9y+fbtSdXKCgoKQnJyMxMRENG7cGPXr10dERATCw8MxatQoLFiwAK1atcLVq1cRGxuLRx55RG+aTqNBgwZo2LAhvv76a/j5+SE1NRUzZsyQtPH29oazszM2bdqExo0bw8nJSW8ZeL169fDCCy9g+vTp8PT0RJMmTbBgwQLk5+fj2WefrfyNIiIRc26I6pBp06bBzs4O7dq1Q6NGjUzmiCxevBgNGjRAz549MWLECERGRqJLly7V2NtSb775JsaMGYNx48YhPDwcrq6uiIyMFJdlm+Oxxx7DkCFDMGDAADRq1Ag//fQTFAoFNmzYgL59+2LixIlo1aoV/vOf/+DSpUvw8fExei2lUok1a9bg8OHD6NChA6ZOnYqPPvpI0sbe3h6ffvopvvrqK/j7+2PkyJEGrzV//nw89thjePrpp9GlSxckJSVh8+bNaNCggdmfjYj0sUIxEdUqarUabdu2xZNPPon33ntP7u4QUQ3EaSkiqtEuXbqELVu2oF+/figoKMDnn3+O5ORkjB07Vu6uEVENxWkpIqrRlEolVq9eje7du6NXr144fvw4tm3bhrZt28rdNSKqoTgtRURERDaFIzdERERkUxjcEBERkU1hcENEREQ2hcENERER2RQGN0RERGRTGNwQERGRTWFwQ0RERDaFwQ0RERHZlP8HZrekEmkySnkAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Testing\n",
        "now that our model has trained, we can apply it to our test data to see how good it is."
      ],
      "metadata": {
        "id": "aGsz--Hq_mvJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "num_true = 0\n",
        "test_quant = 0\n",
        "\n",
        "for (x_test, y_test) in zip(test_data.data, test_data.targets):\n",
        "    #running test example through model\n",
        "    y_pred_test = model(x_test.type(torch.float32))\n",
        "\n",
        "    #getting predicted value (highest value)\n",
        "    y_pred_test = torch.argmax(y_pred_test)\n",
        "\n",
        "    # print('---')\n",
        "    # print(y_test)\n",
        "    # print(y_pred_test)\n",
        "\n",
        "    #checking to see if the right value was predicted\n",
        "    if bool(y_pred_test == y_test):\n",
        "        num_true += 1\n",
        "    test_quant += 1\n",
        "\n",
        "print(f'Accuracy %:')\n",
        "print((num_true/test_quant)*100)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yYCI_5Qn_ZIm",
        "outputId": "859ce9c0-8ebd-4124-d523-ae052903e01d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy %:\n",
            "96.69\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Ia7052BYOVxP"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}